Skip to content
llmkit

go · multi-vendor · one-method interfaces

Twelve AI providers, one model name away.

Pick a model — gpt-5, claude-sonnet-4-5, gemini-2.5-pro, llama3.2:3b — and llmkit routes it to the right vendor. You code against Chatter, Streamer or Embedder, each a single method, and swap providers without touching a call site. Pure Go 1.27, no cgo, stdlib plus one dependency.

Install Go Reference

MIT licensed · streaming, tools, images, audio, PDFs, embeddings, rerank

main.go 
chat, err := llmkit.Open[llmkit.Chatter]("claude-sonnet-4-5")
resp, err := chat.Chat(ctx, &llmkit.Request{
    Messages: []llmkit.Message{llmkit.UserText("Explain iter.Seq2")},
})
fmt.Println(resp.Text())

// same code, different vendor
chat, err = llmkit.Open[llmkit.Chatter]("openrouter/openai/gpt-4o")

what you get

Small interfaces, honest capabilities.

Every provider exposes exactly the methods its API supports, so a missing capability fails at construction — not three network calls in.

One-method interfaces

Chatter, Streamer, Embedder, Reranker. Ask for exactly what you need with Open[T]; compose them into your own interface when you need more.

Model-name routing

Bare names resolve by prefix. Ambiguous ones take provider/model. Open-weight names fall back to a local Ollama daemon — or whatever fallback you set.

Streaming as iterators

Stream returns iter.Seq2[Chunk, error]. The request is sent when you range, closed when you break, and always ends with one finish chunk carrying usage.

Tool calling

Declare tools with JSON Schema, get typed ToolCalls back, or let RunTools drive the loop until the model stops asking.

Multimodal input

Text, images, audio and PDFs are typed message parts. Each provider maps what it accepts and rejects the rest with ErrUnsupported before sending anything.

Escape hatches

Extra and ProviderOptions merge raw fields into the wire body; every response keeps its raw JSON. Register your own OpenAI-compatible server in three lines.

providers

Native wire formats, no vendor SDKs.

Every REST provider is written against its API with net/http. Six OpenAI-compatible vendors share one transport; Vertex AI over gRPC ships as a separate nested module so its dependency tree stays opt-in.

OpenAI

Responses API. Chat, stream, tools, images, PDFs, reasoning, embeddings.

Anthropic

Messages API. Tools, images, documents, extended thinking, structured output.

Vertex AI

REST and gRPC. Gemini chat, images, audio, video, PDFs, tools, thinking, embeddings.

Ollama

Native API. Local models, images, tools, thinking, embeddings. The default fallback.

Cohere

v2 API. Command chat, tools, vision, thinking, embeddings and rerank.

VoyageAI

Text and multimodal embeddings, rerank.

DeepSeek · Groq · x.ai

OpenAI-compatible chat and streaming with tools; reasoning content surfaced as parts.

OpenRouter · Hugging Face

Hundreds of hosted models behind one key each; embeddings included.

Yours

vLLM, llama.cpp, LM Studio — any OpenAI-compatible endpoint via openaicompat.NewProvider.

usage

Stream it, or let it call your tools.

Two idioms cover most applications: range over a stream, or hand RunTools a map of functions.

// Stream any model, print deltas as they arrive
stream, _ := llmkit.Open[llmkit.Streamer]("llama3.2")
for chunk, err := range stream.Stream(ctx, req) {
    if err != nil { return err }
    if chunk.Kind == llmkit.ChunkText {
        fmt.Print(chunk.Text)
    }
}
// Tool loop: runs until the model stops calling tools
tools := map[string]llmkit.ToolFunc{
    "weather": func(ctx context.Context, args json.RawMessage) (string, error) {
        return lookup(args), nil
    },
}
resp, err := llmkit.RunTools(ctx, chat, req, tools, 5)
  • Open[T]
  • New
  • ParseModel
  • Register
  • RunTools
  • Collect
  • WithAPIKey
  • WithBaseURL
  • WithHTTPClient

install

Add it to your module.

Requires Go 1.27+. The gRPC Vertex provider is a separate module you opt into.

go get go get github.com/richardwooding/llmkit
optional go get github.com/richardwooding/llmkit/vertexgrpc
cli go install github.com/richardwooding/llmkit/cmd/llmkit@latest

Full API on pkg.go.dev. Keys come from the usual environment variables (OPENAI_API_KEY, ANTHROPIC_API_KEY, …).