Skip to content
hostrate

go · per-host · http roundtripper

Polite HTTP, one host at a time.

A tiny http.RoundTripper that rate-limits outbound requests per host, so being slow to one server never throttles another. The map[host]*rate.Limiter pattern from the Go wiki — done once, properly, and composable.

Install Go Reference

MIT licensed · one file, one dependency · Go 1.26+

client.go 
client := hostrate.NewClient(2, 5) // 2 req/s per host, burst 5
resp, err := client.Get("https://example.com/feed.xml")
// a slow host never starves requests to any other host

what you get

Per-host politeness, without the boilerplate.

A plain http.RoundTripper that gives every host its own token bucket — no shared global budget, no background goroutine, nothing to close.

Per-host token buckets

Each host gets its own limiter, created lazily — one slow host never throttles another. No shared global budget across all your remotes.

Composable

It's a plain http.RoundTripper. Wrap any base transport and drop it into any http.Client — it slots into what you already have.

Context-aware

Throttled waits honor request deadlines and cancellation — a slow bucket won't outlive the context that started the request.

Bounded memory

Optional idle eviction (WithIdleTimeout) reclaims limiters for hosts you stop talking to — safe for long-running crawlers.

Flexible keying

Default KeyByHost, plus KeyByHostPort or any custom WithKeyFunc — key by host, host:port, or per API token.

Small & focused

One file — stdlib plus golang.org/x/time/rate. No background goroutine, nothing to Close.

usage

A ready client, or your own transport.

Start with NewClient for a batteries-included *http.Client, or build a RoundTripper around your own transport with the options you need.

// Ready-made client — 2 req/s per host, burst 5
client := hostrate.NewClient(2, 5)

resp, err := client.Get("https://example.com/feed.xml")
// each host is throttled independently
// Wrap your own transport as a RoundTripper
rt := hostrate.New(myTransport, 2, 5,
    hostrate.WithIdleTimeout(10*time.Minute),
    hostrate.WithKeyFunc(hostrate.KeyByHostPort),
)
client := &http.Client{Transport: rt}
  • NewClient
  • New
  • WithBase
  • WithIdleTimeout
  • WithKeyFunc
  • KeyByHost
  • KeyByHostPort

install

Add it to your module.

One import, one dependency. Requires Go 1.26+.

go get go get github.com/richardwooding/hostrate
import import "github.com/richardwooding/hostrate"

Full API on pkg.go.dev.