Skip to content
archives

go · streaming · no cgo

Walk every file inside an archive.

Point it at an APK, a .deb, or a firmware.tar.gz and get an io.Reader for every contained file — descending through nested containers automatically, with decompression-bomb guards.

Install Go Reference

MIT licensed · pure Go, no cgo · descends nested containers

archives.go 
archives.Walk(ctx, "pkg.deb", archives.Options{},
  func(ctx context.Context, path string, r io.Reader) error {
    n, _ := io.Copy(io.Discard, r)
    fmt.Printf("%-32s %d bytes\n", path, n)
    // pkg.deb!/data.tar!/usr/bin/htop  123456 bytes
    return nil
  })

what you get

Every leaf, streamed and bounded.

One recursive walker that descends into nested containers, hands you each file as a stream, and refuses to be blown up by hostile input.

Streaming

Files are handed to your callback as io.Readers; the whole archive is never loaded into memory.

Recurses automatically

Descends nested containers — ar → tar.xz → ELF, zip → dex — down to every leaf.

Virtual paths

Each leaf gets a !/-separated path so you can locate findings deep inside nested containers.

Magic-byte detection

Format is detected from content, not extension; non-archives simply emit as a single leaf.

Bomb-guarded

MaxDepth (8) and MaxBytes (2 GiB) bound untrusted input; it stops gracefully, not with an error.

Pure Go

Stdlib plus pure-Go zstd/xz decoders, no cgo; context cancellation is observed mid-stream.

usage

Basic walk, or bounded and instrumented.

Start with a zero-value Options{} and a callback. Opt into depth/size limits and a logger when the input is untrusted.

// Basic — walk every leaf
err := archives.Walk(ctx, path, archives.Options{}, fn)
// Safety + opt-in diagnostics
err := archives.Walk(ctx, path, archives.Options{
    MaxDepth: 8,
    MaxBytes: 2 << 30,
    Logger:   slog.Default(),
}, fn)
  • zip
  • apk
  • jar
  • tar
  • gzip
  • bzip2
  • xz
  • zstd
  • deb

install

Add it to your module.

One import, pure Go, no cgo.

go get go get github.com/richardwooding/archives
import import "github.com/richardwooding/archives"

Powers txtr's --recurse. Full API on the Go Reference.