Models and routing

Claude models go through /messages, everything else through /chat/completions, picked from the live /models catalog (cached 24h). Codex models use the Responses API over SSE with automatic token refresh.

Role chains in .zerd/config.toml:

[routing]
default   = ["claude-sonnet-5-5", "deepseek/deepseek-v4-pro"]
fast      = ["deepseek/deepseek-v4.1-flash"]       # easy requests
strong    = ["claude-opus-5-5"]                    # hard ones, and escalation
summarize = ["deepseek/deepseek-v4.1-flash"]       # context compaction
review    = ["gpt-5.6-sol"]
advisor   = ["deepseek/deepseek-v4-pro"]

Difficulty tiers. Each request is classified without a model call: small, well-defined work goes fast; debugging, concurrency, security, architecture and multi-file work go strong. A failed repair or a loop escalates to strong. -m or /model pins a model. --effort low|medium|high or /effort low|medium|high|auto sets how hard it thinks.
Per-family sampling and output caps. Temperature and top_p follow the model family unless you set temperature. max_tokens is clamped to the window and learned from a provider's 400 message; a reply cut off by the limit is continued, and a tool call cut mid-arguments is never run.
Thinking switches. Easy requests turn thinking down, hard ones up, in each server's own wording: DeepSeek thinking.type, z.ai thinking (with clear_thinking off), OpenRouter reasoning.effort, Together reasoning.enabled, chat_template_kwargs on your own vLLM/SGLang/LM Studio, reasoning_effort for gpt-oss. A server that refuses the switch gets the request again without it.
zerd probe-models shows which of your configured models actually answer.
zerd insights [--days 7] [--json] reads zerd's own trace files and shows, per model, how many calls failed or were retried, how much of the prompt came from cache, how many tool calls were malformed, and how often the loop had to step in (empty reply, tool call written as text, cut-off reply, loop). Per tool it shows the error rate and the most common error. It ends with a short "worth a look" list, so prompts and compat settings get tuned from what happened, not from memory.
Any provider, directly. provider:model skips the gateway and calls that API with its own key: zerd -m groq:llama-3.3-70b, -m anthropic:claude-sonnet-5-5, -m ollama:qwen3:8b. Built in: openai, anthropic, gemini, openrouter, groq, deepseek, mistral, xai, together, fireworks, cerebras, moonshot, zai, ollama, lmstudio. Set the key (GROQ_API_KEY=...), or type /connect groq: it asks for the key, saves it to zerd's .env, and lists the models for /model groq:<Tab>. These ids also work in [routing] chains, so a fallback can live on a different provider. Anything else that speaks the OpenAI or Anthropic format takes three lines:
```
[providers.work]
base_url    = "https://llm.internal.example.com/v1"
api_key_env = "WORK_LLM_KEY"   # leave out for a local server that needs no key
protocol    = "anthropic"      # default "openai"
```
A server that is only mostly OpenAI-compatible (llama.cpp, vLLM, SGLang, a company proxy) is tuned the same way, no code:
```
[providers.box.compat]
max_tokens_field     = "max_completion_tokens"   # or "none" to send no cap
stream_usage         = false                     # the server rejects stream_options
tool_result_name     = true                      # tool messages must carry the tool's name
assistant_after_tool = true                      # the chat template refuses a user message right after a tool result
cache_control        = true                      # Anthropic cache marks (on by itself for anthropic/* on OpenRouter)
extra_body           = { thinking_token_budget = 4096, chat_template_kwargs = { enable_thinking = true } }
```
A model that garbles several tool calls in one reply can be held to one at a time with extra_body = { parallel_tool_calls = false }. A reply that carries its thinking inline as <think>...</think> is split automatically: the thinking is kept out of the answer.

extra_body is merged into every request last, so it also overrides zerd's own thinking switch for that server. A thinking budget matters on local models: thinking and the answer share max_tokens, and without a cap the thinking can use all of it and leave no tool call.

A project's own .zerd/config.toml can only set [providers] once you trust it, because it could otherwise send your keys to its own server.