Skip to content
Docs menu

Orchestrator API

The Go orchestrator's OpenAI-compatible and admin HTTP API (78 endpoints): routes, headers, bodies and errors.

View .md

Generated from the HTTP API section of docs/architecture.md, the orchestrator's normative contract. The OpenAI-compatible routes take Authorization: Bearer <client key> and work with the official openai SDKs; the admin API takes Authorization: Bearer $VENOM_ADMIN_TOKEN. Each endpoint below has a stable anchor, #ep-<method>-<path>.

OpenAI-compatible surface (Authorization: Bearer <client key>)

  • GET /v1/models — OpenAI list format; contains every route alias (e.g. venom-default) plus the concrete upstream/model ids that can be served: each upstream's listed models and its route targets, except a route target whose model the upstream doesn't have (see "Model availability" below), which is hidden so a client never picks an id that can only fail. Each entry may carry a non-standard context_window (tokens); for an alias it is the smallest known window among the targets that can serve right now (healthy or not yet checked, breaker not open, model available), or among all enabled targets when none can. It follows upstream health, so it can change between calls: with Ollama down, venom-default advertises OpenAI's window rather than Ollama's. An alias is listed unless every one of its targets is on a disabled upstream (glm with neither QWEN_API_KEY nor ZAI_API_KEY set): nothing could answer it, so a client choosing by this list (the CLI's gears, Venom Cloud's gear picker for a daemon's chats) never starts on it, and a request for it still gets 503 no_upstream naming the missing keys. When some of a listed alias's targets lack their model it carries a non-standard unavailable: [{upstream, model, reason, fix}] (omitted otherwise), and when some of its enabled targets are on an upstream that isn't local (neither a self-hosted kind nor a loopback/private/single-label address) a non-standard cloud_targets: ["upstream/model", …] (omitted otherwise). venom doctor reads both: the second keeps it from reporting a loopback orchestrator as keeping prompts on the machine when its routes fall back to a cloud. Every alias also carries a non-standard capabilities: {"vision": bool}: true only when every target it can reach accepts image input (the intersection over the same targets context_window uses: those that can serve now, else every enabled one; a target whose capability is unknown counts as text-only). An upstream/model entry carries it when the orchestrator knows: the config's capabilities.vision, Ollama's /api/show capabilities, or a known vision family by id (gpt-4o/4.1/5, o1/o3/o4-mini, claude, gemini, qwen-vl, llava, …; quirks.KnownVision); otherwise it is omitted. The CLI turns image input on from it (venom --chrome screenshots). Because the alias flag follows health, a client can hold vision: true from while a text-only target was down; a request that carries an image (an image_url or input_image part) therefore skips the targets known to be text-only (configured vision: false, or Ollama's capabilities) whenever another target remains. A route may also name an images: route (glm → glm-vision, qwen-coder → qwen-vision): a request that carries an image is then served by that route's targets instead of its own, and the alias reports the images route's vision, so a text-only model's alias still takes screenshots. While no target of the images route can serve, the alias reports its own targets' vision, and an image request goes to those of its own targets not known to be text-only, else gets 400 image_input_unavailable. The request log's route is the images route when it served. The list object also carries a non-standard top-level profiles: the configured route profiles' names, sorted ([] when none), so a client can tell whether an x-venom-profile it would send exists (venom's gears are unavailable when theirs is missing). The model list itself ignores x-venom-profile: aliases are listed with their configured targets' windows and vision, and one a profile overrides with [] is still listed. Each entry may also carry a non-standard max_inflight: how many requests to that id the orchestrator runs at once. An upstream/model entry has its upstream's max_inflight; an alias has the sum over the distinct upstreams of its enabled targets (an upstream counted once however many of its models the route names). It is omitted when unbounded (any of those upstreams sets none) or when the alias has no enabled target. The CLI sizes its scheduler's model pools from it (ADR 0009 §2.1), so its agents queue in the client rather than at the upstream's semaphore. The CLI sizes its context from the list and refreshes it at session start. A key whose allowed_models names only an alias still gets both annotations on it, by decision (2026-10-09): every answer already names the target that served it (x-venom-upstream, x-venom-model), a 503 model_unavailable names each target and its reason, and cloud_targets tells the key holder where their prompts can go, which they are owed more than the operator's routing is owed secrecy. What a scoped key does not get is the upstreams' concrete ids as list entries.

  • GET /v1/models/{model} — one entry of that list (the SDKs' models.retrieve; the / of an upstream/model id may be sent raw or as %2F), with the same visibility (a key's allowed_models, hidden unavailable targets); otherwise 404 {"error":{"code":"model_not_found"}}.

  • POST /v1/chat/completions — streaming (SSE) and non-streaming.

  • POST /v1/completions, POST /v1/embeddings — routed the same way; bodies are passed through after model rewriting. Embeddings: input (a string or an array; token arrays where the upstream takes them), dimensions and any extension field (input_type) pass through. encoding_format: "base64" (the OpenAI SDKs' default when the caller names none) is passed on; Ollama is asked for floats, and the answer of every kind but openai and vllm is encoded here when it comes back as floats (float32 little-endian, base64, as OpenAI encodes it), so openai-node's default call works against any upstream. Kind bedrock serves embeddings through InvokeModel: Titan Text Embeddings V2 (amazon.titan-embed-text-v2:0, one input per request; an array of more is a 400 unsupported_request) and Cohere Embed (cohere.embed-*, a batch; input_type default search_document), usage from X-Amzn-Bedrock-Input-Token-Count. A usage object with only total_tokens (Voyage, Jina) is read as prompt tokens.

  • POST /v1/rerank — rerank documents against a query (the Cohere / Jina shape). Request: {"model", "query": string, "documents": [string | {"text": string}], "top_n"? (or Voyage's "top_k"), "return_documents"?: bool, "max_tokens_per_doc"?, "instruct"?} (1–1000 documents). Answer: {"id": the request id, "object": "list", "model": the served model, "results": [{"index", "relevance_score", "document"?: {"text"}}], "usage": {"prompt_tokens", "total_tokens", "search_units"?}}, results best first and cut to top_n; the documents come back only with return_documents: true, from the request (an upstream is never asked to echo them). Routed like any other endpoint: aliases (venom-rerank), upstream/model, fallback, breakers, retries, strategy: cheapest, client-key auth, RPM/TPM, budgets and the request log (tokens in, 0 out, the price table's input per 1M tokens plus search_unit per search). Each upstream gets it in its dialect (rerank: on the upstream, else by kind): jina (POST {base_url}/rerank: vLLM, llama.cpp, SGLang, Jina, Together; the default), cohere (base_url https://api.cohere.com/v2; meta.billed_units read), voyage (top_k, data), tei (Hugging Face text-embeddings-inference: POST {origin}/rerank {query, texts}, its /v1 dropped), dashscope (Model Studio's native POST {origin}/api/v1/services/rerank/text-rerank/text-rerank, the default for kind dashscope: qwen3-rerank, gte-rerank-v2), and kind bedrock through InvokeModel (cohere.rerank-v3-5:0, amazon.rerank-v1:0, billed per query of up to 100 documents). A reranker that reports no tokens (TEI, Bedrock) is logged at the query-and-documents estimate (bytes/4). Statuses: 400 invalid_rerank_request (no query, no documents or over 1000, top_n < 1, a document that is neither a string nor {text}), a reranker's own error relayed in the OpenAI error shape ({"message"} and Model Studio's {"code","message"} become {"error":{…}}), the rest as for chat. Never streamed. An alias whose every target model has "rerank" in its name, and any such upstream/model, carries endpoints: ["/v1/rerank"] in /v1/models.

  • POST /v1/systemone — typed decisions in TypeSafe's System One wire format [R285]: {"state": string|object|array, "model": "venom-decide", "questions": {"<id>": {"type": "noul"|"choice"|"score", "instructions": …, "criteria"?: …}}} in, the upstream's {"model": "jev-1.13.0", "answers": {"<id>": …}, "usage": {"input_tokens", "output_tokens"}} back unchanged (x-venom-upstream, x-venom-model added). Never streamed. Same client-key auth, RPM/TPM limits, budgets and request log as the other /v1 routes. model is resolved like any other: the venom-decide alias (→ typesafe/jev-latest), or an upstream/model id such as typesafe/jev-1.13.0. Only decision models serve it (an upstream of kind typesafe, or any kind with capabilities.decisions: true), and they serve nothing else. Body cap: 2 MiB (proxy.MaxDecisionBodyBytes, or max_body_bytes if smaller). Statuses: 400 invalid_decision_request (no state, questions not a non-empty object, or stream: true), 400 unsupported_endpoint (the model is a chat model, or a chat endpoint was asked of a decision model), 404 model_not_found, 413 request_too_large, 401/429 from the client key's auth and limits, 503 no_upstream naming the missing credential (typesafe: TYPESAFE_API_KEY is not set). Upstream answers: 2xx relayed unchanged with usage logged and priced (input tokens only, config/prices.yaml); 422 (TypeSafe's validation error) relayed unchanged; 429 and 529 (overloaded) retried with backoff inside the retry budget, then relayed with their retry-after headers; 401/403 from TypeSafe → 502 upstream_auth_failed (the orchestrator's key is wrong, not the client's). A decision alias and its upstream/model ids carry a non-standard endpoints: ["/v1/systemone"] in /v1/models (chat clients skip them), and a decision alias with no enabled target is not listed at all (like any alias). An alias whose every target model has "embed" in its name (venom-embed), and any such upstream/model, carries endpoints: ["/v1/embeddings"] the same way, one whose every target is a reranker ["/v1/rerank"], and one whose every target model has "whisper" or "transcribe" in it carries endpoints: ["/v1/audio/transcriptions"].

  • POST /v1/audio/transcriptions — OpenAI's speech to text, a multipart/form-data body (file, model, optional language, prompt, response_format, temperature, …). model is resolved like any other (alias with fallback, upstream/model), with the same client-key auth, limits, budgets, breaker and request log. The body is spooled to a temp file (TMPDIR), capped at max_body_bytes, and deleted when the request ends; the orchestrator reads only the model part (in any position, the SDKs send file first), and each attempt streams the file to the target with only that value replaced by the target's model, so fallback resends the whole form. Every other part goes through byte for byte; the reply (JSON, text, or SSE for a streamed transcription) is relayed unchanged. Statuses: 400 invalid_multipart (not multipart/form-data with a boundary, or malformed before model), 400 missing_model, 404 model_not_found, 413 request_too_large; an Anthropic target can't serve it (unsupported_endpoint). The upstream must answer GET /v1/models for its health check (vLLM and speaches do, whisper.cpp's whisper-server doesn't). The reply's usage {"type": "duration", "seconds"} is logged as audio_ms and priced at the model's per_second. A target on a kind with a native speech-to-text adapter (Voice, below) gets the form translated to its provider's API and the answer translated back to the response_format asked for.

  • POST /v1/audio/translations — OpenAI's speech to English text: the same multipart body, spooling, routing and statuses as transcriptions, relayed to {base_url}/audio/translations. A native adapter that doesn't translate is skipped (400 unsupported_endpoint when no target can).

  • Voice (ADR 0044, internal/voice, internal/voiceapi). The contract every voice adapter, client and console page builds on:

    • POST /v1/audio/speech — OpenAI's text to speech, JSON: {model, input, voice?: "<id>" | {"id": "<id>"}, response_format?: mp3 (default) | opus | aac | flac | wav | pcm, speed?: 0.25–4.0, instructions?, stream_format?: "audio" (default) | "sse", stream?: bool, venom_options?: {...}}. model resolves like any other (alias with fallback, profile, upstream/model) and only targets that are text-to-speech models are tried (capabilities.tts, a voice kind's default, or a known id: tts-1*, gpt-*-tts, Kokoro, Piper, Orpheus, CosyVoice, MiniMax speech-*, ElevenLabs eleven_*, …). The answer is the audio, 200 with the format's Content-Type (audio/mpeg, audio/ogg, audio/aac, audio/flac, audio/wav), written and flushed as the provider sends it: never buffered, and a target is given up for the next only before its first byte (503, 429, no answer → next target; the provider's other 4xx relayed as JSON). stream: true asks an adapter whose provider has a separate streaming endpoint to use it; the client-side shape is the same either way. stream_format: "sse" answers text/event-stream with OpenAI's events, data: {"type": "speech.audio.delta", "audio": "<base64>"} per chunk and data: {"type": "speech.audio.done", "usage": {…}} last: passed through from an upstream that produces them (OpenAI's gpt-4o-mini-tts), built by the orchestrator around any other (its done usage is {"input_characters": n}). pcm is raw signed 16-bit little-endian mono samples, 24 kHz unless the adapter (or the request's venom_options.sample_rate, where the provider honours it) says otherwise, and the answer always names its rate: Content-Type: audio/pcm;rate=24000;channels=1 and x-venom-sample-rate: 24000 (the header also on wav, and on an SSE answer of either). The realtime voice runtime (ADR 0043) reads it from there. venom_options is venom's one extension object (extra_body in the SDKs): language (BCP-47 hint, for providers that take one), sample_rate (Hz, pcm and wav where the provider lets you pick), provider (an object merged last, at the top level, into the native request body the adapter sends: any provider-specific option, e.g. {"voice_settings": {"stability": 0.4}} for ElevenLabs). To an OpenAI-compatible upstream the client's body goes as sent, with model replaced, stream and venom_options removed and venom_options.provider's keys added. Headers x-venom-upstream, x-venom-model. Usage: the input's characters (Unicode code points), logged as audio_chars and priced at the model's per_1m_chars; key budgets reserve the worst target's cost by characters (a budgeted key can't reach an unpriced cloud TTS model: 403 unpriced_model). Statuses: 400 invalid_json, missing_model, missing_input, invalid_voice, invalid_response_format, invalid_speed, invalid_stream_format, model_not_tts (no target speaks); 403 model_not_allowed / profile_forbidden; 404 model_not_found; 413 request_too_large (body over 1 MiB); 429 budget_exceeded / upstream_rate_limited; 502 upstream_error. A stream that breaks after the first byte is logged 502 upstream_broken (499 client_closed when the client left).
    • GET /v1/audio/voices?model=<alias | upstream/model> — {"object": "list", "data": [{"id", "name", "languages"?: [...], "gender"?, "preview_url"?, "provider": "<upstream name>", "model": "<upstream>/<model>"}]}: the union of the voices of every enabled TTS target of the model, in target order. Voice ids are the provider's (an alias falling back to another provider needs a voice that one knows). Kind openai answers OpenAI's built-in voices (alloy, ash, ballad, cedar, coral, echo, fable, marin, nova, onyx, sage, shimmer, verse); other OpenAI-compatible servers are asked GET {base_url}/audio/voices (Kokoro-FastAPI) then GET {base_url}/audio/speech/voices (Speaches), and a server with neither lists none. A target whose list fails is left out and named in x-venom-voices-failed; all failing is 502. preview_url is the provider's CDN: the orchestrator never fetches it. Statuses: 400 missing_model, 400 model_not_tts, 404 model_not_found. Not logged.
    • Adapters: one per provider wire format, registered by upstream kind (voice.Register; ADR 0044). A kind without one is served by the OpenAI-compatible adapter (openai, vllm, sglang, llamacpp, ollama, generic; Speaches, Kokoro-FastAPI, LocalAI): zero translation. A provider kind declares its traits with config.RegisterKind: audio only (its models serve /v1/audio/* only; chat to one is 400 unsupported_endpoint), tts/stt defaults, its auth header, and no model list (models must be listed). Kind names: elevenlabs, minimax (shared with chat), dashscope (Qwen-TTS, CosyVoice, Paraformer), byteplus-speech, fishaudio, mistral (Voxtral, shared with chat); local servers cosyvoice, qwen3-tts, indextts, sparktts, fishspeech.
    • Native cloud adapters (checked against each provider's docs 2026-11-03; all three audio only, TTS + STT, models must be listed). A format a provider can't produce, or a missing required voice, is 400 with code unsupported_by_provider before any call. A native transcription answers json, text, verbose_json, srt or vtt (built from the provider's timed transcript); none of the three translates.
      • elevenlabs — base_url https://api.elevenlabs.io/v1, key in xi-api-key. Speech POST /text-to-speech/{voice} (…/stream when stream: true), output_format mp3_44100_128, opus_48000_64, pcm_<rate> / wav_<rate> (8000–48000; no aac, flac), speed → voice_settings.speed, language → language_code; voice required. Voices GET /v2/voices?page_size=100. Scribe POST /speech-to-text (multipart, model_id, word timestamps grouped into sentences, audio_duration_secs billed). Health GET /models.
      • fishaudio — base_url https://api.fish.audio, Authorization: Bearer. Speech POST /v1/tts with the model in the model header (s1, s2-pro, s2.1-pro, drama-3), format mp3/opus/wav/pcm (wav/pcm at 8, 16, 24, 32 or 44.1 kHz; no aac, flac), voice → reference_id (optional), speed → prosody.speed, language. Voices: your own models (GET /model?self=true), else the 50 most used public ones. ASR POST /v1/asr (multipart audio, model header transcribe-1 / transcribe-1-pro, duration billed).
      • byteplus-speech — base_url https://voice.ap-southeast-1.bytepluses.com, key in X-Api-Key (a Seed Speech console key). Speech POST /api/v3/tts/unidirectional (model as X-Api-Resource-Id: seed-tts-2.0, seed-tts-1.0, seed-icl-2.0; the fixed X-Api-App-Key), mp3/opus/pcm; its stream of {code, data: base64} objects is decoded to raw audio as it comes, and an error code in the first object is a refusal (4xx relayed). speed → speech_rate (−50..100), language → additions.explicit_language; voice (a speaker id) required. Voices: a static subset of the official TTS 2.0 list (no list API). Speech to text: the flash recognizer POST /api/v3/auc/bigmodel/recognize/flash (model seed-asr-fast or volc.seedasr.auc_turbo; wav, mp3 or ogg files, base64 in JSON, encoded as it is sent). Streaming ASR and bidirectional TTS are WebSocket only (the realtime path, ADR 0043); the standard recognizer takes URLs only and isn't offered.
    • Native adapters on kinds that also chat (checked against each provider's docs 2026-11-03):
      • dashscope (Qwen TTS: qwen3-tts-flash): POST {api}/services/aigc/ multimodal-generation/generation ({api} is the …/compatible-mode/v1 base_url with /api/v1 in its place), {model, input: {text, voice, language_type}}, Authorization: Bearer, always X-DashScope-SSE: enable. Each event's output.audio.data is base64 of the next piece of one streamed WAV (24 kHz mono s16le; verified live); wav relays it, pcm relays it without the header; other formats are 400 unsupported_response_format. voice defaults to Cherry; venom_options.language → language_type; speed and instructions are not sent. Voices: the documented static list of 45 system voices. Error events map to statuses (InvalidParameter* 400, Throttling* 429, InvalidApiKey 401, else 502). CosyVoice and Paraformer / qwen-asr are WebSocket APIs: realtime (ADR 0043), not here.
      • minimax (T2A v2: speech-2.8-hd, speech-2.8-turbo, legacy 2.6 / 02 / 01): POST {base_url}/t2a_v2 (https://api.minimax.io/v1), {model, text, voice_setting: {voice_id, speed}, audio_setting: {format, sample_rate, channel: 1}, language_boost}, Authorization: Bearer. Without stream one JSON answer with hex data.audio; with stream: true or stream_format: "sse" the event stream (hex per event, exclude_aggregated_audio: true). MiniMax answers errors as HTTP 200 with base_resp.status_code: 1002/1039 → 429, 1004 → 401, 1008 → 402, 2013/1026/1027 → 400, 1000/1001/1024/1033 → 500, else 502. Formats mp3, pcm, flac, wav, opus (aac 400); voice required (400 missing_voice); speed clamped to 0.5–2; venom_options.language → language_boost, sample_rate one of 8000–44100 (24000 for pcm/wav by default). Voices: POST {base_url}/get_voice {"voice_type": "system"}. The same upstream serves MiniMax chat; list the speech models beside the chat ones.
      • mistral (Voxtral): POST /v1/audio/transcriptions is OpenAI's multipart shape, so no adapter: an upstream of kind mistral (or generic) at https://api.mistral.ai/v1 with voxtral-mini-latest passes through; its usage.prompt_audio_seconds is logged as audio_ms. In every native adapter venom_options.provider is merged last into the provider's body: an object value is merged one level deep into an object already there ({"voice_setting": {"emotion": "calm"}} keeps voice_id), anything else replaces it.
    • Local TTS servers (wire formats read from each project's source on main, 2026-11-03; kinds registered with KindTraits.Local, so they cost $0 wherever base_url points; audio only, TTS, models must be listed, no key needed (an api_key_env goes out as Authorization: Bearer)). Voice cloning: venom_options.reference_audio is a data:audio/…;base64,… URL, an http(s) URL (handed to an upstream that fetches it itself, refused for one that doesn't: the orchestrator never fetches it), or a file path read from the upstream's reference_audio_dir (refused when unset; .., absolute paths elsewhere and symlinks leaving the directory are 400). The clip must be WAV, MP3, FLAC or Ogg by its magic bytes and at most 10 MiB (a data: URL is also bound by the 1 MiB request body); venom_options.reference_text is its transcript.
      • cosyvoice — CosyVoice's FastAPI server (runtime/python/fastapi/server.py, port 50000; base_url is its root). Multipart form fields to /inference_sft {tts_text, spk_id} (a voice), /inference_instruct (+instruct_text, CosyVoice 1), /inference_zero_shot {tts_text, prompt_text, prompt_wav} (a clip with its transcript), /inference_cross_lingual (a clip alone), /inference_instruct2 {tts_text, instruct_text, prompt_wav} (a clip with instructions). CosyVoice 3 (cosyvoice3 in the model id) gets You are a helpful assistant.<|endofprompt|> ahead of the transcript or instruction; <|endofprompt|> is appended to instructions. The answer is headerless s16le mono PCM at 22050 Hz (model ids with 300M: CosyVoice 1) or 24000 (2 and 3); wav gets a streamed header in front, other formats 400. No speed, voice list or health route. venom_options.provider keys become form fields.
      • qwen3-tts — Qwen3-TTS open weights served by vLLM-Omni (vllm serve <model> --omni; the Qwen3-TTS repo has no HTTP server), base_url http://host:8091/v1. POST /audio/speech with OpenAI's fields plus language (from venom_options.language: Chinese, English, …), sample_rate, stream: true + stream_format: "audio" for stream (pcm/wav only; sse passed through), task_type: "Base" + ref_audio + ref_text (or x_vector_only_mode) when a clip is given, task_type: "VoiceDesign" for a VoiceDesign model (instructions required). instructions is the model's instruct. Rate from x-audio-sample-rate, else 24000. Voices GET /audio/voices ({voices: [...], uploaded_voices: [{id, name}]}).
      • indextts — IndexTTS2 / 2.5 served by vLLM-Omni (the index-tts repo's README route; base_url …/v1). The same POST /audio/speech with ref_audio from the clip (required unless voice names a voice uploaded to the server), instructions → extra_params: {emo_text, use_emo_text: true}, venom_options.language → extra_params.lang (zh, en, ja, es, ar); emo_vector, emo_alpha, emo_audio through venom_options.provider.extra_params. 22050 Hz. Voices as qwen3-tts.
      • sparktts — Spark-TTS on Triton Inference Server (runtime/triton_trtllm; the repo has no REST server). POST /v2/models/{model}/infer with Triton v2 JSON tensors reference_wav (FP32 [1, N]), reference_wav_len, reference_text, target_text; the answer's FP32 waveform is converted to s16le, 16 kHz, pcm or wav (decoded whole: Triton's JSON is one object). A clip is required: a 16 kHz 16-bit or float WAV, first channel, at most 60 s. The shipped model only clones; gender, pitch, speed in venom_options.provider become extra BYTES inputs for a repository that declares them.
      • fishspeech — the self-hosted Fish Speech / OpenAudio API server (tools/api_server.py, port 8080; base_url its root). POST /v1/tts JSON {text, format: wav|pcm|mp3|opus, reference_id (voice), references: [{audio: base64, text}] (the clip), streaming}; streaming is wav only, so a streamed pcm request streams wav with the header taken off. 44.1 kHz (opus 48 kHz); no speed or language field. Voices GET /v1/references/list; health GET /v1/health; optional bearer key. Not Fish Audio's cloud API (fishaudio: model header, prosody, sample_rate).
      • Ollama needs no adapter (text to speech announced in v0.40.11, Linux and Windows in v0.40.13): kind ollama passes /v1/audio/speech through (Ollama makes wav or pcm only, 24 kHz, speed 1, no instructions; its /v1/audio/transcriptions takes WAV). llama.cpp's llama-server has POST /v1/audio/transcriptions (passes through) but speaks on POST /tts (float32 pcm), not /v1/audio/speech, so it isn't a TTS target yet.
    • Capabilities: capabilities.tts / capabilities.stt (per upstream or in model_info) in orchestrator.yaml; /v1/models reports capabilities: {tts, stt} on speech and transcription models and aliases, and endpoints: ["/v1/audio/speech"] on a speech model or an alias whose every target speaks (venom-speak).
    • Venom Cloud serves the same four paths on its own providers (the org's keys, then the platform's), never relayed to daemons; its /v1/models lists a voice model with endpoints ["/v1/audio/speech", "/v1/audio/voices"] or ["/v1/audio/transcriptions", "/v1/audio/translations"]. Uploads are capped at 4.5 MB there (docs/cloud-mesh.md §7.7.9).
  • Realtime transcription (ADR 0043, internal/realtime, the adapters in internal/voice; design in docs/voice-agents.md §5.1). GET /v1/realtime?intent=transcription[&model=<alias>] is a WebSocket (subprotocol realtime when offered) speaking OpenAI's realtime transcription events in front of any streaming speech-to-text backend. Auth as on every /v1 route: Authorization: Bearer <key>, or OpenAI's browser-style subprotocol openai-insecure-api-key.<key> (what the SDKs' OpenAIRealtimeWebSocket sends); a key in the query string is 400 key_in_url. Before the upgrade: 400 unsupported_intent (any intent but transcription), 426 websocket_required, and, when model is in the URL, the routing refusals (403 model_not_allowed / profile_forbidden, 404 model_not_found, 400 model_not_realtime when no target streams, 429 budget_exceeded). Key RPM limits, profiles and key budgets apply; a budgeted key's session reserves a minute of its priciest target's audio.

    • Session. The server sends session.created at once (before routing; transcription null). The client's first message must be session.update within 5 s (else close 4408; anything else, 4400): {type: "session.update", session: {type: "transcription", audio: {input: {format: {type: "audio/pcm", rate: 8000 | 16000 | 24000 (default)} | {type: "audio/pcmu"} | {type: "audio/pcma"}, transcription: {model, language?, languages?, prompt?, keywords?}, turn_detection: null | {type: "server_vad", silence_duration_ms?, threshold?}}}}, venom?: {partials: "replace"}} (venom also accepted inside session). Without model in the URL the session routes on transcription.model. The orchestrator then opens the upstream session, falling back across the alias's targets (breaker, max_inflight; a target that fails its dial or its own session start is skipped and counted), and answers session.updated with the session as accepted (transcription.model is the alias asked for) plus venom: {upstream, model}. Audio the client sends before that waits in the socket. Later session.updates get error session_locked.
    • Client events: input_audio_buffer.append {audio: base64} (at most 1 MiB decoded, else error audio_chunk_too_large), input_audio_buffer.commit (ends the item; error input_audio_buffer_commit_empty when nothing was appended), input_audio_buffer.clear.
    • Server events (each with an event_id): input_audio_buffer.committed {item_id, previous_item_id} (for client commits and for items the backend's VAD ended), input_audio_buffer.cleared, input_audio_buffer.speech_started {item_id, audio_start_ms} / speech_stopped {item_id, audio_end_ms} (backends that run VAD), conversation.item.input_audio_transcription.delta {item_id, delta} (append-only: for a backend whose partials are revisable, only text it marks stable), venom.transcription.partial {item_id, text, stable_len} (when venom.partials is replace: the whole current hypothesis; stable_len is the prefix, in Unicode code points, that won't change), conversation.item.input_audio_transcription.completed {item_id, content_index, transcript, venom?: {confidence?, words?: [{text, start_ms, end_ms}], language?}} for every item, …failed {item_id, error}, and error {error: {type, code, message, retryable}}. Items are named item_<request id>_<n> and complete in order.
    • Close codes: 1000 normal; 1008 policy (model not allowed, budget); 1011 the upstream failed after session.updated (error with retryable: true first; the client reconnects and the orchestrator may pick another target); 1013 no upstream could start the session (retryable); 4400 bad first message or model; 4408 no session.update within 5 s.
    • Never buffered. Each append is decoded, converted (G.711 / PCM16 at 8, 16, 24 kHz, internal/audio) and handed to the upstream writer at once; each upstream event is translated and written at once. Buffers are reused (no allocation per frame). An upstream that stops taking audio never stalls the client: past 1 s of queued audio frames are dropped and counted (dropped_ms in the orchestrator's log line for the session), and a commit it can't take within 1 s ends the session with 1011.
    • Formats. Each adapter lists the formats its backend takes; the client's format is used when it is one of them, else the audio is converted to the adapter's first.
    • Log and price. One request-log row per session, written at close (status 200 for a normal close, else the close's HTTP equivalent: 400 for 4400, 408 for 4408, 403 or 429 for 1008, 502 for 1011, 503 for 1013, and 499 when the client vanished): stream: true, latency_ms the session's length, ttft_ms the first transcript, audio_ms the audio the backend billed (when it reports it) or the audio sent to it, priced at the model's per_second (0 for a local upstream).
    • /v1/models: an alias whose targets all stream carries endpoints: ["/v1/realtime?intent=transcription"] (both that and /v1/audio/transcriptions when its models also take multipart transcriptions) and capabilities.stt.
    • Kinds: funasr (FunASR's funasr-wss-server-2pass, base_url: ws://host:10095, no key; audio only, no model list, its health check a WebSocket open and close); openai ({base_url}/realtime?intent= transcription, the session passed through with the model rewritten: gpt-live-transcribe and gpt-realtime-whisper take no server VAD, so turn_detection goes as null; gpt-live-transcribe and gpt-transcribe get languages and keywords, the others language and the keywords in prompt; G.711 passes through, PCM goes at 24 kHz; items are re-ordered when completions arrive out of order); dashscope (Model Studio, the WebSocket root derived from base_url: qwen3-asr-flash-realtime over /api-ws/v1/realtime, OpenAI-style events with an event_id on each and a revisable stash; paraformer-realtime-v2, paraformer-realtime-8k-v2 (Beijing only) and fun-asr-realtime over /api-ws/v1/inference run-task sessions, a commit finishing the task); elevenlabs (scribe_v2_realtime, {base_url}/speech-to-text/realtime, commit_strategy manual or vad, µ-law passed through as ulaw_8000; no clear, so a cleared item is committed and its transcript dropped); whisperlive (WhisperLive, base_url: ws://host:9090, no key, 16 kHz int16; it has no flush, so a commit sends reset_transcript and the acknowledgement ends the item with the trailing segment as last seen; health is a WebSocket open and close). Compose profile asr runs a FunASR server on 127.0.0.1:10095. Venom Cloud does not relay WebSockets: realtime goes to a local orchestrator or a reachable daemon.
  • Venom Line's media bridge (ADR 0061 §7, internal/commsmedia, VEN-277). GET /v1/comms/media is a WebSocket a phone provider streams a live call to; it is not behind client auth (the provider holds no key) and exists only with VENOM_COMMS_MEDIA_SECRET. The credential is the cloud's ticket, vml1.<base64url(JSON {c: call, o: org, n: nonce, e: unix expiry, p: sinch|twilio|fake, r: 8000|16000})>.<base64url(HMAC-SHA256(secret, "vml1." + payload))>, valid 120 s and single use (nonces kept until expiry), in ?ticket= (Sinch's stream URL, checked before the upgrade: 403 bad_ticket), or in the first message (Sinch's connect headers venom-ticket, Twilio's start customParameters.ticket; close 4408 without one, 4403 when bad). Also 404 comms_media_not_configured, 426 websocket_required, 503 too_many_calls (VENOM_COMMS_MAX_CALLS).

    • Setup. The bridge posts the ticket to {VENOM_COMMS_CLOUD_URL}/api/integrations/comms/line/session with X-Venom-Line-Proof: base64url(HMAC-SHA256(secret, "session." + ticket)) and gets the line's settings and a session token (vls_…, held only by the bridge); then POST {VENOM_COMMS_SERVE_URL}/v1/calls with transport: "bridge", output_rate (the provider's rate), context (the org's instructions as line_instructions) and line (opening, voice, tools, max_call_s, max_usd), and opens the returned media socket. Sinch then gets {"command":"answer"}.
    • Media. Sinch [R2090]: binary PCM16 LE at the connectStream rate (16 kHz asked) both ways, clear; Twilio Media Streams [R1708]: base64 µ-law 8 kHz media, mark, clear, dtmf, stop. Caller audio goes to the agent as PCM16 16 kHz (µ-law decoded and resampled with internal/audio), agent audio comes back as PCM16 at the provider's rate and is re-encoded. Nothing is buffered: each frame is converted and written as it arrives, in per-call buffers (no allocation per frame in the bridge's code). Marks are Twilio's own; for Sinch the bridge answers them from a playout clock (audio sent, played in real time). Agent clear (barge-in) goes straight to the provider.
    • Tools and transfer. The agent's tool {id, name, arguments} and transfer {reason} go to …/comms/line/tools with the session token; the answers come back as tool_result / transfer_result.
    • After the call the bridge reads GET /v1/calls/{id} and /transcript from venom serve and posts the outcome, the caller-heard transcript, cost and latencies (end of speech to first audio and barge-in p50/p95 from the voice loop, the bridge's own per-frame µs in and out) to …/comms/line/report.
    • The hosted media edge that relays to a daemon over its mesh connection is not built: an org runs its own orchestrator with a public wss:// ingress (docs/public/comms/venom-line.md).
  • POST /v1/images/generations — OpenAI's image API: {model, prompt, n?: 1–10, size?: "WxH" | "auto", response_format?: "url" | "b64_json", …} → {created, data: [{url | b64_json, revised_prompt?}], usage?}. model resolves like any other (alias with fallback across targets, upstream/model); a target is a kind sdapi backend (AUTOMATIC1111 / Forge / SD.Next / stable-diffusion.cpp server, POST {base_url}/sdapi/v1/txt2img, which also takes negative_prompt, steps, seed, cfg_scale, sampler_name) or an OpenAI-compatible one ({base_url}/images/generations: OpenAI, LocalAI). Backends are always asked for base64; url (the default) is a link into the orchestrator's own file store, /v1/files/{id}/content?exp=&sig=, that works without a key for an hour (the file is the key's, purpose image_generation, kept 24 h). Errors: 400 (missing_prompt, missing_model, invalid_n, invalid_response_format, a bad size), 403 model_not_allowed, 404 model_not_found, a backend's own 4xx relayed, 502 upstream_error when every target failed. One request-log row each. /v1/models marks image models and aliases endpoints: ["/v1/images/generations"].

  • Batches and files (OpenAI's Batch API; internal/batch). Files and batches belong to the client key that made them (another key gets 404; open mode: one shared owner).

    • POST /v1/files — multipart/form-data with file and purpose (batch, user_data, assistants, vision, evals, fine-tune), at most 200 MB, streamed to $VENOM_DATA_DIR/files/ → the file object {id: "file-…", object: "file", bytes, created_at, filename, purpose, status: "processed"}. GET /v1/files?purpose=&limit=&after= (a list object, newest first), GET /v1/files/{id}, GET /v1/files/{id}/content (the bytes), DELETE /v1/files/{id} (409 file_in_use while an unfinished batch reads it). Batch results are files of purpose batch_output.
    • POST /v1/batches {input_file_id, endpoint, completion_window: "24h", metadata?} → the batch object (id: "batch_…", status, request_counts, output_file_id, error_file_id, the *_at times, errors), as OpenAI's. endpoint is /v1/chat/completions, /v1/completions, /v1/embeddings or /v1/responses; the input file must have purpose batch. Each input line is {custom_id, method: "POST", url: <endpoint>, body}; validation (unique custom_id, the batch's endpoint, a model, no stream, ≤ 50,000 lines) fails the batch with line-numbered errors. GET /v1/batches/{id}, GET /v1/batches?limit=&after=, POST /v1/batches/{id}/cancel (cancelling, then cancelled; 409 batch_not_cancellable once it ended). Output lines are {id, custom_id, response: {status_code, request_id, body}, error: null}: 2xx in the output file, other statuses in the error file, and a line that never got a response (cancelled, expired, its key deleted) as {…, response: null, error: {code, message}}. A batch past its 24 h window ends expired with what finished.
    • Execution: one batch at a time, oldest first. Each line is served in-process as the batch's key through the same handler as /v1 (routing, fallback, budgets, the request log), with x-venom-priority: background (the background queue at a full upstream, flex on OpenAI) and x-venom-session: <batch id>, so the Sessions view sums the batch's cost. Up to 4 lines at once while no foreground request is in flight, 1 otherwise; while idle, a line for an alias whose first target is in the cloud goes to the alias's first servable local target. A 429 or 5xx is retried up to 5 times with backoff. On an upstream with native_batch: true (kind openai), a batch whose every line resolves there is uploaded to the provider's own Batch API instead (/files, /batches, polled every 30 s, cancel forwarded); its output and error files are copied back and each line is logged at half the list price. State (files, batches, per-line results) is in SQLite, so a restart resumes a batch where it stopped. native_batch also takes kind dashscope (Model Studio's OpenAI-compatible batches) and kind anthropic (Message Batches, chat completions only: lines translated like a synchronous request, results back as chat completions). Native batches are polled alongside the batch running here.
  • Provider tiers and batch requests (x-venom-service-tier, x-venom-urgency).

    • x-venom-service-tier: auto | default | flex | priority asks for that tier where the body has no service_tier, over a route profile's and the background lane's (default = standard, outright). OpenAI: sent as is (flex only on flex models; a flex 429 is sent again at once on the default tier). Anthropic: priority and auto → auto, default → standard_only, flex → none. Other kinds: nothing. Any other value → 400 invalid_service_tier. The answer carries x-venom-service-tier: <tier sent>. Flex and priority are priced at their flex: / priority: rates; the request log keeps service_tier and saved_usd (standard price minus billed), summed by GET /admin/v1/spend → tiers: [{tier, requests, cost_usd, saved_usd}].
    • x-venom-urgency: batch (or x-venom-service-tier: batch) on /v1/chat/completions, /v1/completions, /v1/embeddings, /v1/responses: the request becomes a batch request. A model whose first target is native_batch is collected with the key's other requests for it (batch_routing.window, default 30 s, or max_lines, default 1000) into one provider batch at half price; anything else (and a request under a route profile, or from a key with a hard spend limit) runs at once on the background lane. Answer: 202 {id: "breq_…", object: "batch_request", endpoint, model, status: queued | in_progress | completed | failed | cancelled | expired, mode: provider_batch | background | "", batch_id, created_at, completed_at, response: {status_code, request_id, body} | null, error: {code, message} | null} with Location: /v1/batch_requests/{id} and x-venom-batch-request: <id>. x-venom-batch-wait: <seconds> (≤ batch_routing.max_wait, default 10 m) waits and answers like the synchronous call (or 202 when it isn't back). stream: true on chat completions: 200 text/event-stream at once, SSE comments every 15 s, then the result as chat.completion.chunk events and [DONE] (other endpoints stream on the background lane, x-venom-urgency: background on the answer). x-venom-callback-url POSTs the finished object there, only to a host listed in batch_routing.callback_hosts (else 400 callback_host_not_allowed). x-venom-urgency: background serves now on the background lane; now changes nothing; anything else → 400.
    • GET /v1/batch_requests?limit=&after= (newest first, without responses), GET /v1/batch_requests/{id}?wait=<seconds> (long-poll up to max_wait), POST /v1/batch_requests/{id}/cancel (while queued; 409 batch_request_not_cancellable after). Owned by the key that made them; kept 7 days after they finish.
  • POST /v1/search — venom's own: web search over the configured backends (search: in orchestrator.yaml, alias venom-search), so search keys stay on the server. Body: {"query": string (required, ≤ 1000 bytes), "model"?: "venom-search" | a target's name (forces that backend), "count"?: 1–10 (default 8; more is capped at 10), "time_range"?: "day"|"week"|"month"|"year", "allowed_domains"?: [host], "blocked_domains"?: [host], "language"?, "answer"?: bool} (answer: true tries the backends that write a grounded answer first: openai, dashscope, tavily). Answer: {"object": "search.result", "query", "model", "backend": the target whose results lead, "results": [{"title", "url", "snippet", "published"?, "source": target}], "answer"?: string, "citations"?: [{"url", "title"}], "attempts": [{"target", "reason", "results", "ms"}], "usage": {"calls", "tokens_in", "tokens_out"}, "cost_usd"}. Targets are tried in order; one without its key, base URL or upstream is skipped with no request; one that fails or finds nothing hands over (reason: 5xx, 429, auth, timeout, bot_check, error, empty, breaker_open after 3 failures in a row, for 60 s); one with fewer than merge_below (4) results is thin and merged once with the next one's. Results are deduplicated (by canonical URL: host without www., path without a trailing slash or index file, tracking parameters such as utm_*, fbclid, gclid, srsltid dropped; then a near-duplicate snippet, 80% of the words shared; then a repeated title on the same host without a snippet), trimmed (skip links, navigation and cookie-banner text, a leading date, the title repeated in the snippet, the site name at the end of a title; tracking parameters and #:~:text= fragments are removed from the returned URL; published is a date), cleaned (control characters, one line, title ≤ 200, snippet ≤ 300 chars cut at a word) and domain-filtered, in the backend's order, so the same backend answer is always the same bytes. Client-key auth and allowed_models (the key must list venom-search, or the forced target), RPM, budgets (pre-charged at the costliest target's fee plus 20k/1k tokens of its model), the request log (route: venom-search, upstream = the leading target, attempt_log, cost_usd = per-call fees plus the grounded backends' tokens at list price) and the spend ledger are the chat path's. Headers: x-venom-upstream, x-venom-model (the target), x-venom-fallback (the reasons of the targets passed over). Errors: 400 invalid_search_request, 403 model_not_allowed, 404 model_not_found (unknown target), 429 budget_exceeded, 502 search_failed (every target failed; the last error named), 503 no_upstream (none enabled, with why). GET /v1/search/backends lists the targets: {name, backend, enabled, reason?, call_usd, answer, model?} (no keys, no addresses).

    backend needs per call (list price, 2026-10-11) answer
    searxng base_url (SEARXNG_URL) $0 (self-hosted) no
    brave / tavily / exa api_key_env $0.005 / $0.008 / $0.005–0.007 tavily
    duckduckgo nothing $0 (HTML scraping, bot checks) no
    openai upstream: a kind openai upstream, model $0.01 per search + tokens (gpt-5.4-mini ≈ $0.017 a query) yes, with url_citations
    dashscope upstream: a kind dashscope upstream, model (qwen-plus; qwen3.8 refuses) tokens + search fee (≤ $0.01, unpublished) yes
    zai upstream: a kind zai upstream, model = engine (search_pro) $0.01 no

    call_usd on a target overrides its fee; a target on a local base URL costs 0.

  • POST /mcp — the same search as an MCP server (Streamable HTTP transport, JSON responses, no SSE stream: GET answers 405; protocol versions 2025-11-25, 2025-06-18, 2025-03-26), so Claude Code, Cursor, Zed and other MCP clients use venom's search and keys with one config entry. Client-key bearer auth as on /v1. Methods: initialize, ping, tools/list, tools/call; notifications get 202. Tools: web_search (query, count, answer, fetch: also the text of the top 0–3 pages, time_range, allowed_domains, blocked_domains; text content plus structuredContent with results, answer, citations, pages; the text is Results for "<query>": then per result N. title (date), the URL and the snippet on their own lines, with no backend name, timing or cost, so identical results are identical text in the agent's context) and fetch_page (url, max_chars ≤ 40,000: a public page as text; scripts, navigation and other chrome dropped). Page fetches refuse loopback, private, link-local, CGNAT and other non-public addresses at connect time (after DNS, redirects included), credentials in URLs and non-text types. venom's own harness calls /v1/search directly, not MCP. Claude Code (.mcp.json):

    {
      "mcpServers": {
        "venom-search": {
          "type": "http",
          "url": "http://localhost:8787/mcp",
          "headers": { "Authorization": "Bearer ${VENOM_API_KEY}" }
        }
      }
    }
  • GET /v1/usage[?session=<id>&sessions=5] — venom's own: the calling key's spend and budgets, for the CLI's /usage and venom usage. It takes no key parameter; the key is the one that authenticated the request (in open mode, the requests logged without a key), so a client can never read another key's figures. Body (object: "venom.usage"): now; key {id, name, team}; budget {daily, monthly}, each {spent_usd, reserved_usd, soft_usd, hard_usd, state: ok|soft|exceeded|none, resets_at} (UTC day / month, as /admin/v1/keys shows it); today, last_7_days (today and the 6 UTC days before it) and month (UTC month to date), each {from, to, requests, tokens_in, tokens_out, cost_usd, cached_tokens, cache_hit_ratio, by_model: [{model, local, requests, tokens_in, tokens_out, cost_usd, cached_tokens}]}, figures from the spend ledger by key id (local upstreams are $0, their tokens counted); top_sessions: today's costliest sessions of this key (sessions, default 5, at most 50), each {session_id, first_ts, last_ts, requests, errors, tokens_in, cached_tokens, tokens_out, cost_usd, subagents, top_model} with its subagent sessions (x-venom-parent-session) rolled in; session: ?session=<id> in the same shape plus models, or null when this key has no logged request in it; log, notes. Cached tokens, local and sessions come from the request log, which records key names: they count only rows logged under the caller's name, and when another key (live, or deleted but still in the ledger) has that name they are left out (log: false, cached_tokens: null, a note) rather than mixed. A session id another key also used shows only the caller's rows. 400 for a bad sessions or a session id over 256 bytes; 401 like the other /v1 routes (the admin token is not a client key).

  • GET /v1/routing[?window=24h&alias=a,b&recent=10] — venom's own: where the calling key's requests to each route alias went, and why, for the CLI's /routes and venom routes. Like /v1/usage it takes no key parameter and reads the request log by the caller's key name (refused, log: false with a note, when another key has that name; open mode: requests logged without a key). window: 1h, 24h (default), 7d, any Go duration or <n>d, 1m to 31d; alias: also list these aliases with no traffic (only ones the key's allowed_models permit); recent: how many latest fallbacks (default 10, at most 100). Body (object: "venom.routing"): now, window, since, scope: "key", key {id, name, team}, log, notes, requests, cost_usd, aliases (most requests first) and recent_fallbacks. Each alias: {alias, strategy, configured, requests, unserved, unserved_share, fallbacks, fallback_share, fallback_reasons: {reason: n}, tokens_in, tokens_out, cost_usd, targets, recent_fallbacks}; each target, in configured order (then targets the log names that a reload removed, position: 0, state: "removed"): {position, upstream, model, configured, state, detail, until, capabilities: {tools, responses, decisions, context_window}, requests, share, errors, tokens_in, tokens_out, token_share, cost_usd, cost_share, latency_ms, ttft_ms, live_ttft, fallbacks_in, fallbacks_in_reasons, passed_over, passed_over_reasons}. state is the target's live state now: ok, half_open (its breaker admits one probe), cooling (a quota 429 cooldown; until), circuit_open (until = when it goes half-open), unhealthy (last health check failed), missing (the upstream lacks the model), disabled (detail names the missing env var), removed. share = requests the target served (any status) / the alias's requests; every target's share plus unserved_share (requests no target served) is 1. latency_ms / ttft_ms are {p50, p90, n} (nearest rank) over the target's successful requests / streams, null without any; live_ttft is the router's own decayed estimate now ({p50_ms, p90_ms, samples, age_s}, as on /admin/v1/routes; null before a sample). fallbacks_in: requests it served after an earlier target did not, by the reason of the first target passed over; passed_over: times it did not serve, by reason; an alias's fallbacks counts requests with at least one target passed over. Requests routed under a route profile (x-venom-profile, or the key's default) are their own block: profile (absent = none) is on the alias block and its fallbacks, and under a profile that re-targets the alias the block's targets are the profile's, in its order. A fallback is {ts, request_id, alias, profile?, from, to, reason, status, chain}: from the first target passed over, to the one that served ("": none did), chain the request's attempt_log. Nothing names an upstream's address or credential. 400 for a bad window or recent.

  • GET /v1/gears — venom's own: the gears the orchestrator's config defines, for the CLI's gear list (see "Gears" below). Same auth as /v1/models. Always 200; without gears: the list is empty.

  • POST /v1/responses — streaming and non-streaming. To a target whose capabilities.responses is true (OpenAI by default) it is passed through. To any other target (Ollama, llama.cpp, vLLM, SGLang and generic servers by default) it is translated to /v1/chat/completions and the reply translated back into a Responses object or Responses SSE events (response.created, response.in_progress, response.output_item.added|done, response.content_part.added|done, response.output_text.delta|done, response.reasoning_text.delta|done, response.function_call_arguments.delta|done, then response.completed / response.incomplete, or error + response.failed; every event carries sequence_number). Translated: input (string or message / function_call / function_call_output / reasoning items, whose text goes on the assistant turn they preceded; text and input_image URL parts), instructions, function tools, tool_choice (incl. allowed_tools), max_output_tokens, temperature, top_p, parallel_tool_calls, text.format, reasoning.effort, user, safety_identifier, prompt_cache_key, service_tier. metadata is echoed on the Response but not sent upstream, and parallel_tool_calls / tool_choice are dropped when no tools remain: chat completions refuses all three in those cases where Responses accepts them. Usage maps to input_tokens/output_tokens (+ cached, cache-write and reasoning details, every field present as the API reference requires). The translation is stateless: a request with previous_response_id or conversation skips translated targets and gets 400 {"error":{"code":"stateful_request_unsupported"}} when no native target remains. Hosted tools (web_search, file_search, …), item_reference inputs, input_file parts and stored prompts likewise can't be translated (400 unsupported_request, or the next native target). Translated responses are never stored (store: false), except background ones (below).

  • Background responses (background: true, proxy/background.go). The route's first target decides how one runs. Native (capabilities.responses): the request is passed through, and the queued reply's id is mapped (SQLite background_responses) to the upstream, its model and the calling key. The queued reply is logged at cost 0 and its budget and intent pre-charges stay reserved; once the response is terminal (completed, incomplete, failed, cancelled), seen on a client's GET or by the poller (every 2 s doubling to 1 min, for up to 6 h; then, or if the upstream forgets it, the pre-charge estimate is booked), its real usage is priced and booked exactly once (budget, spend ledger, intent) and written onto the POST's request-log row. A streamed background request that runs to its end is booked then, like any stream. In mesh mode the row also keeps the cloud ticket or relay job the POST came under (mesh_ref), and the settled usage goes to the cloud as a usage report with settle: true (docs/cloud-mesh.md §6.6). Emulated (any translated target): the orchestrator answers the queued Response itself (its own resp_… id, background: true, store: true) and runs the request detached as a translated stream; it is billed like any request when it ends (a cancel bills what was generated, like a client hanging up) and its final Response is kept for 24 h. With stream: true the POST streams the run's events; the run goes on if the client leaves. A background request's request-log row has background: true (the console's bg badge). Mappings of native responses are kept 30 days.

  • GET /v1/responses/{id} — a background response this key created (another key's, or an unknown id: 404 not_found). Native: proxied to its upstream with the upstream's credentials, query string included. Emulated: the stored object; ?stream=true replays its events from the start, or after starting_after=<sequence_number>, and follows a running one to its end (after 10 min in memory only the terminal event remains).

  • POST /v1/responses/{id}/cancel — native: proxied (the cancelled response settles at the usage the upstream reports for it). Emulated: aborts the run and answers the cancelled object; a completed one answers 400 invalid_state.

  • DELETE /v1/responses/{id} — native: an unsettled response is read (and cancelled if it still runs) and settled first, then the delete is proxied. Emulated: cancels a running one, then {"id","object":"response","deleted":true}.

A model value resolves as: route alias → its target list (an experiment alias → the target of the variant the request is assigned, see "Model A/B experiments"); upstream/model → that upstream directly; bare model id → first upstream advertising it. An id that contains / but that its prefix upstream can't serve (disabled, or not in its static models list) is looked up as a bare id, so vLLM's openai/gpt-oss-20b still reaches vLLM.

Gateway errors use the OpenAI {"error":{message,type,code}} shape. They name the upstream but never its address or the transport error (dial tcp 10.0.0.5:11434: …); that detail goes to the log, last_error and the admin request log. An upstream answering 401/403 means the orchestrator's credential for it is wrong: the upstream is marked failed, the next target is tried, and if none serves the client gets 502 upstream_auth_failed (the upstream's own body, which can echo part of the key, is not relayed). A request body is read within 60 s (408 request_timeout otherwise) and bounded by max_body_bytes (413 request_too_large). A client that stops reading a response is dropped once one write to it has waited defaults.write_stall_timeout (60 s; time waiting on the upstream doesn't count) and its upstream request cancelled. A stream ends at its terminal event (data: [DONE], or a Responses response.completed, response.incomplete or response.failed): the orchestrator closes the client's stream there, reading at most 1 s more for the upstream's own end, so an upstream that keeps the connection open can't hold the client. An upstream that sends nothing for defaults.stream_idle_timeout (5 min) in the middle of a response is cut, and a stream ends with its error event. A client that closes a stream after the upstream finished it ([DONE], or a finish_reason plus usage) is logged with the upstream's status, not 499.

Response headers added: x-venom-request-id, x-venom-upstream, x-venom-model (the concrete model that served the request), x-venom-base-model (the model that one is priced as, when its config sets a base_model), x-venom-translated: chat_completions on a translated /v1/responses reply, and x-venom-variant: <variant> on every answer to an experiment alias (errors included; absent for any other model), and x-venom-profile: <name> on every answer routed under a route profile (errors after the profile is chosen included), x-venom-response-cache: hit | shared on a side request answered from the response cache (the original's headers otherwise, with this request's own id), x-venom-dedup: joined | replayed on a request answered from an identical one's upstream call or a stored Idempotency-Key response (the latter also with Idempotent-Replayed: true; → Request deduplication). x-venom-fallback: <reason>[,<reason>…] is on an answer the first target it tried did not serve: the reason code of each target passed over or failed, in order (429_quota, quota_cooldown,timeout); absent when the first target served. Reason codes (stable): 429 (rate limited), 429_quota (quota or balance exhausted; the target then cools down), quota_cooldown (skipped while cooling), circuit_open, unhealthy (skipped: last health check failed), busy (at max_inflight), unsupported (the target can't serve this request: tools, endpoint), context_window (passed over: the request's estimated tokens, prompt bytes/4 + max output, exceed its context_window, so a larger one serves; the response also carries x-venom-context-cost-usd, the estimated input-cost difference in USD, served minus passed over, when both are priced; nothing is ever truncated), missing (the upstream lacks the model, or answered 404 for it before), timeout (no first byte within ttft_timeout / the upstream timeout), 5xx, 404 (404/501), auth (the upstream refused our credential), conn_error, context_length (the upstream answered 400 context_length_exceeded; only a target with a larger or unknown window is tried next, and if none serves, that 400 goes to the client), rejected_param (the upstream answered 400 for a field or value it doesn't take, e.g. reasoning_effort must be one of low/high/max, unsupported_parameter, Model Studio's InvalidParameter, Z.ai 1210/1212/1214/1215: the next target is tried and the 400 is held for the client if none serves; a 400 whose body says the request doesn't parse is relayed at once), rate_shaped (passed over: at its provider's learned rate limit while a later target could serve now, or its rate-shaping queue refused the wait). When no target's context_window holds the request (prompt bytes/4

  • its max output tokens), the answer is an OpenAI-shaped 400 before any attempt: {"error":{"message":…,"type":"invalid_request_error","param":"messages","code":"context_length_exceeded"}} (param is input on /v1/responses, prompt on /v1/completions). When some target fits but none serves (503 no_healthy_upstream, 502 upstream_unavailable), the message names every target of the route, not only those tried or skipped as down: after the attempts' reasons come the targets that can't take this request with why (… but ollama/qwen3:8b has a 4096-token context window, plus the OLLAMA_CONTEXT_LENGTH fix for an Ollama target) and the targets on a disabled upstream (openai: OPENAI_API_KEY is not set), all ; -separated. x-venom-upstream and x-venom-model are set on every committed answer (chat completions, responses, translated responses, /v1/systemone) before the first byte, since the fallback is decided by then. The CLI's call meter reads both on every request it makes, streamed or not, to show which model answered each call (cli/AGENTS.md → "Call meter"); per-request cost is priced client-side from config/prices.yaml, since a header can't carry a cost known only at the end of a stream. Every /v1 answer, errors included, also carries x-request-id, where the OpenAI SDKs read a request id from: the same value as x-venom-request-id, unless the upstream sent its own x-request-id, which is relayed instead. Upstream rate-limit headers (retry-after, retry-after-ms, x-ratelimit-*) are relayed with an upstream 429, so the SDKs' retry timing follows them. Upstream response headers pass by allowlist and nothing else does: content-type, content-length, content-encoding, cache-control, x-request-id, retry-after, retry-after-ms, x-should-retry, x-ratelimit-*, openai-processing-ms, openai-version (what the OpenAI SDKs read, plus what describes the body). Cookies, hop-by-hop headers, account ids (openai-organization, openai-project), server banners and an upstream's own x-venom-* never reach the client. Retries happen at one layer at a time (see "Routing semantics"): a request carrying x-stainless-retry-count > 0 (an OpenAI SDK's own retry) gets no same-upstream retry here, and an upstream 429/503 asking for a longer wait than defaults.backoff_max is returned at once instead of slept on, so an SDK call reaches an upstream at most max_attempts + max_retries times (unless rate shaping queues that wait, below). The OpenAI SDKs are checked against this surface by make conformance (services/go-orchestrator/AGENTS.md, "Conformance").

Rate shaping (rate_shaping: in orchestrator.yaml; services/go-orchestrator/AGENTS.md → Rate shaping): each upstream (one provider credential) learns its provider's limits from the x-ratelimit-{limit,remaining,reset}-{requests,tokens} and anthropic-ratelimit-{requests,tokens}-{limit,remaining,reset} headers of its answers and from its 429s (retry-after-ms / Retry-After; 500 ms when a 429 says nothing). A request that would go over a learned limit is sent to a later target that has room now, else waits for the limit to refill in that upstream's queue, ordered by start-time fair queuing across client keys (and mesh teams), at most rate_shaping.max_wait (default 30s) with at most rate_shaping.max_queue (default 256) waiting. A request that waited carries x-venom-queued-ms: <ms>. A provider 429 the shaper learned from is retried after waiting in that queue rather than a backoff, also when its Retry-After is longer than backoff_max but within max_wait (and the retry budget's time). Only when the queue is full or the wait would pass the bound does the client get 429 {"error":{"type":"rate_limit_error","code":"upstream_queue_full"|"upstream_wait_exceeded",…}} with Retry-After / Retry-After-Ms (when it would fit, estimated), before any upstream call. rate_shaping.weights: {<key name>: <weight>} (default 1, at most 1000) gives a key a larger share of a contended upstream. max_queue: -1 or max_wait: -1s turns waiting off: the learned limits then only steer requests to targets with room, and the last target is sent the request as before. An upstream that never sends a rate-limit header (local engines) is not shaped.

Rate limits per client key (rpm_limit, tpm_limit): each request takes one RPM token. Before it is forwarded, its estimated size (prompt bytes/4 + the requested max_tokens / max_completion_tokens / max_output_tokens) is charged to the key's TPM bucket, and afterwards the charge is reconciled to the usage the upstream reported (unused tokens refunded; a request that produced no response is refunded in full). A request that doesn't fit yet gets 429 rate_limit_exceeded with Retry-After; one larger than the whole per-minute limit gets 429 tpm_limit_exceeded (no Retry-After).

Spend budgets per client key (budget, set via the admin API; a key without one is unlimited). Periods are the UTC calendar day and month and reset at 00:00 UTC. After routing, each request is pre-charged its worst-case cost: prompt bytes/4 plus its max output (4,096 tokens when it sets none; 0 for embeddings) at the list price of the costliest candidate it could be served by. Then:

  • spent + in-flight pre-charges + this pre-charge > a hard limit → 429 with {"error": {"type": "insufficient_quota", "code": "budget_exceeded", "message": …}}, x-should-retry: false (the OpenAI SDKs raise RateLimitError and don't retry) and x-venom-budget-reset: <RFC 3339 UTC>. No Retry-After. A request whose pre-charge is 0 (every candidate local) is never refused, so local models keep working past a budget.
  • a key with a hard limit, and an enabled cloud candidate the price table has no price for → 403 {"error": {"type": "permission_error", "code": "unpriced_model"}}: its cost would be booked as $0, so the budget couldn't limit it. Add the model to config/prices.yaml. A key with only soft limits or no budget is served (its spend there reads $0).
  • spent ≥ a soft limit → the request is served and the response carries x-venom-budget-warning (the period, its spend and limits, when it resets).

When the request finishes, its pre-charge is released and its actual cost (see Cost) is booked. A response whose usage never arrived (the client hung up on a stream before its usage chunk, the upstream broke mid-response, or sends none) is booked at an estimate: prompt bytes/4 plus the output the events' text adds up to (usage.TextTokens: content, reasoning and tool-call argument strings at about 4 bytes a token, at least one per event that carried text; one per event when none did; body bytes/4 when not streamed), capped at the request's max output, so cancelling a stream just before its end isn't free, and a provider that streams many tokens per chunk isn't billed as one. The request log carries those estimated tokens with the cost. Venom Cloud does the same for what it serves itself (lib/providers/partial.ts) and books a relayed job's partial end at the daemon's figures (docs/cloud-mesh.md §6.5). Spend is kept per key in memory (O(1), no allocation on the request path), in a spend ledger in SQLite that is not pruned with the request log, and is reloaded from the ledger at start.

Request deduplication

Identical concurrent requests from one client key (same endpoint, route profile, Accept and OpenAI-Beta, and the same body with whitespace removed; a mesh team's requests never meet another team's) share one upstream call: a client's retry or double submit while the first is still generating. Eligible: temperature 0 set explicitly, /v1/embeddings and /v1/rerank (unset temperature is the provider's default, 1 for OpenAI, so never), or any request with an Idempotency-Key (≤ 255 bytes, else 400 invalid_idempotency_key). Never a stateful or background /v1/responses request or an audio upload. X-Venom-Dedup: off opts a request out.

  • The response is fanned out as it streams, never buffered whole: each client reads it at its own pace, and one more than dedup.max_lag_bytes (256 KiB) behind is cut loose (its connection reset, as a broken upstream's would be; before anything was written to it, it goes upstream on its own), so a slow client never stalls the others. A duplicate can join while the response is at most dedup.max_bytes (1 MiB) and no further along than max_lag_bytes; later, it is sent upstream itself. The first client hanging up doesn't stop a call a duplicate is reading; when every client is gone the upstream request is cancelled.
  • Billed once: the first request's row carries the tokens and cost; each duplicate's row has attempt_log: dedup:joined:<first request's id>, no tokens and cost_usd 0, and its answer x-venom-dedup: joined with its own request id.
  • Idempotency-Key: the same key and body is collapsed at any temperature, and after the call completes it is replayed whole for dedup.idempotency_ttl (5 min) with x-venom-dedup: replayed and Idempotent-Replayed: true (attempt_log: dedup:replayed:<id>). A 5xx, a 429, a broken stream or one over max_bytes is not kept, so a retry goes upstream again. The same key with another body is 422 idempotency_key_reused. Kept responses take at most dedup.idempotency_max_bytes (32 MiB, least recently used dropped).
  • Counted in venom_dedup_total{how}, logged (request deduplicated). Venom Cloud does the same for relayed jobs (docs/cloud-mesh.md §4.1.1).

Intent spend accounts

A 24/7 agent needs a budget per task, not per period. A request that carries X-Venom-Intent: <id> (at most 128 bytes; longer is 400 invalid_intent) is metered against that intent and against its key's team (internal/intents). An intent's cap is a running total, not a UTC day or month, and the same pre-charge as the key budget applies (worst case over the candidates, so concurrent requests can't jointly overshoot). A request over a cap is refused before any attempt:

  • 402 {"error": {"type": "insufficient_quota", "code": "intent_budget_exceeded", "message": …}} with x-should-retry: false. The message names the scope (intent or team), the spend and the cap. A request whose pre-charge is 0 (every candidate local) is never refused.
  • an intent or team with a cap, and a candidate the price table has no price for → 403 unpriced_model, as for a key budget.
  • more than 100,000 distinct intents → 503 intent_ledger_full.

The first refusal of an intent stamps it exceeded (exceeded_at) and logs intent over its cap. GET /admin/v1/intents?state=exceeded is that feed: it is what an inbox item for the owner is built from. Raising the cap above the spend clears the mark, so the next overrun is a new item. An intent is created by its first request (uncapped, owned by the first key's team that names it) or by PUT; accounts persist in SQLite (intents, team_caps).

  • GET /admin/v1/intents[?state=ok|uncapped|exceeded] → {intents: [{id, team, cap_usd, spent_usd, reserved_usd, requests, state, first_at, last_at, exceeded_at}], teams: [{team, cap_usd, spent_usd}]}, exceeded first.
  • GET /admin/v1/intents/{id}, DELETE /admin/v1/intents/{id} (the team keeps the spend counted so far).
  • PUT /admin/v1/intents/{id} {"cap_usd": 5, "team": "core"} sets the cap (0 removes it) and creates the account.
  • PUT /admin/v1/teams/{team}/intent-cap {"cap_usd": 50} caps the total spend of all the team's intents.

Spend is booked when a request finishes, from the same cost as the request log. Not covered: a team's spend counts only requests that name an intent, and several orchestrators on one database keep separate running totals.

Guardrails (guardrails: in the config, guardrails.md): every /v1 inference request is checked against the org's rules, its key team's and its key's own. A block is 403 {"error": {"type": "guardrail_blocked", "code": "GR-ORG-CRED-101", "message", "layer", "action", "url", "request_id"}}; a redaction, warning or keep-local match passes with one x-venom-guardrail: <code>; action=…; layer=…; url=… entry per rule, and a stream ends with one more event listing them (venom_guardrails in a chunk, venom.guardrails on the Responses API). GET /g/{code} (no key) is the code's page. Venom Cloud answers the same way.

Admin surface (Authorization: Bearer $VENOM_ADMIN_TOKEN)

All JSON, snake_case.

  • GET /admin/v1/overview → {uptime_s, requests_total, errors_total, tokens_in, tokens_out, cached_tokens, cache_write_tokens, cache_breaks, cache_hit_ratio, cost_usd, latency_p50_ms, latency_p95_ms, ttft_p50_ms, ttft_p95_ms, active_streams, by_model:[{model, requests, errors, tokens_in, tokens_out, cost_usd}], wasted_tokens, wasted_usd, wasted:[{model, reason, attempts, tokens, usd}]} (since start, in memory; wasted* is the wasted spend of /admin/v1/requests waste, summed by requested model and reason)
  • GET /admin/v1/anomalies → {anomalies:[{scope:"key"|"session", id, name, metric:"spend"|"cache_breaks", value, baseline, ratio, requests?, breaks?, since, last}]}: open cost anomalies, newest first (in memory). value and baseline are $/hour for spend and a break ratio (0–1) for cache_breaks; ratio is 0 while the entity has no baseline yet. Rules: services/go-orchestrator/AGENTS.md → Cost anomalies.
  • GET /admin/v1/upstreams → [{name, kind, base_url, disabled, healthy, circuit:"closed"|"open"|"half_open", inflight, latency_ewma_ms, models:[string], last_error, local, retries:{budget_available, first_attempts, retries, skipped:{budget, retry_after, client_retrying}}, quota_exhausted:[{model, until, reason}], load:{waiting, running, kv_cache_usage, at, saturated}, shaping:{…as GET /admin/v1/shaping}}]. shaping (omitted until the upstream sent a rate-limit header or a 429) is its rate shaper (see Rate shaping above). load (omitted when not scraped or stale) is the engine's own queue and KV-cache use (0–1, -1 = not exported) read from its /metrics (vllm, sglang, llamacpp; metrics_url), and whether routing counts it saturated (services/go-orchestrator/AGENTS.md → KV-cache pressure). quota_exhausted (omitted when empty) lists the models sitting out a cooldown after a "quota exhausted" 429, with when they are tried again (RFC 3339). disabled is true for an upstream that is structurally off (its api_key_env is unset): it is never health-checked and healthy is false, but it is not down. local is true when its tokens cost 0 (see Cost below). retries is the upstream's retry budget since start, the same figures as the venom_upstream_attempts_total, venom_upstream_retries_skipped_total and venom_upstream_retry_budget_available metrics: retries the budget allows now, first attempts and retries sent, and retries not made because the budget was spent, the upstream's Retry-After was longer than backoff_max, or the client was retrying itself
  • GET /admin/v1/shaping → {max_queue, max_wait_ms, upstreams:[{name, active, learned_at, requests:{limit, remaining, reset_at}, tokens:{limit, remaining, reset_at}, blocked_until, queue_depth, max_depth, admitted, queued, cancelled, rejected:{queue_full, wait_exceeded}, wait_p50_ms, wait_p95_ms, wait_max_ms}]}: every upstream's rate shaper. requests / tokens (omitted until learned) are the provider's limit and what remains now (refilled linearly to the limit by reset_at, less what was sent since); blocked_until (omitted when past) is a 429's Retry-After; admitted went through at once, queued after waiting, cancelled left the queue (client gone), rejected were answered 429 here; the waits are of the last 256 queued requests.
  • GET /admin/v1/routes → [{alias, strategy, targets:[{upstream, model, weight, available, reason, fix, servable, context_window, ttft, capabilities:{tools, tool_choice, responses, reasoning_field, replay_reasoning}}]}]. ttft is the target's live time-to-first-token estimate, {p50_ms, p90_ms, samples, age_s} (omitted before its first streamed answer): what strategy: fastest and balanced rank by (docs/configuration.md → Latency-aware routing). available is false when routing skips the target because its upstream doesn't have the model; reason then says why and names the fix (model qwen3:8b is not pulled on ollama: run `ollama pull qwen3:8b` / `make pull-models` ) and fix is the fix alone (ollama pull qwen3:8b (or make pull-models)). Both are "" when available. servable is true when routing would send a request to the target now (available, its upstream enabled and healthy or not checked yet, breaker not open); the harness resolves an alias to the first servable target to pick the edit tool. context_window (tokens, configured else discovered; omitted when unknown) and capabilities are the target's spec as the capability gate sees it: kind defaults, then config, then discovery (responses: false means a /v1/responses request is translated; reasoning_field is "" when the upstream emits none; replay_reasoning is true when earlier assistant turns' reasoning is sent back to it in reasoning_field, false when it is removed from requests). The console's model page shows them.
  • GET /admin/v1/routing[?window=24h&key=NAME&alias=a,b&recent=10] → the GET /v1/routing body over every key's requests (scope: "all", no key; ?key=NAME narrows it to one key name), listing every configured alias (with or without traffic), and each fallback with its key. The console's Routing page. It also carries learned (admin only, read-only): for each alias with strategy: learned, the cached per-target scores the router orders by (params, and per target sessions, successes, success_rate, wilson_lower, mean_cost_usd, measured, qualified, rank), plus updated_at and window. Learned routing scores session outcomes against cost; merge and revert rates are a follow-up pending a commit-to-session join and a task-type key (services/go-orchestrator/AGENTS.md → Learned routing).
  • GET /admin/v1/requests?limit=100 (max 1000) → [{id, ts, key_name, model, route, upstream, upstream_model, status, stream, latency_ms, ttft_ms, tokens_in, tokens_out, cached_tokens, attempts, error, session_id, cost_usd, local, parent_session_id, variant, attempt_log, profile, cache_write_tokens, cache_break, wasted_tokens, wasted_usd, waste?:[{upstream, model, reason, tokens_in, tokens_out, usd}], base_model, price_source, role, gear, tools, cache_saved_usd, audio_chars?, audio_ms?}] (newest first). audio_chars is a /v1/audio/speech request's input characters and audio_ms the audio a transcription or translation was billed for (migration 20; omitted when 0). role is the spend lane the request ran as (x-venom-role: the gear roles workers, smart, vision, browser, classifier, summary, reflex, or a side request's lane outside a session turn: review, triage, judge, decide, hook, ask; any other value and a main turn are ""), gear its gear (x-venom-gear, trimmed and lower-cased; kept only when it is a letter then letters, digits, ., _, -, at most 64, as Venom Cloud reads it; else "") and tools whether it offered tools (a non-empty tools array). All three come from migration 15; older rows read "", "", false. model and route are what the client asked for, upstream / upstream_model what served it (after any fallback), base_model the model that one is priced as ("" = itself) and price_source where cost_usd's price came from: custom, base_model, builtin, local or unpriced (both "" on rows before migration 13 and for requests nothing served). waste lists the attempts of the request that were billed and discarded (absent when none), with wasted_tokens / wasted_usd their sums: reason is abandoned (the upstream had started answering and the attempt was dropped, or it broke a committed response), ttft_timeout (cancelled at the TTFT deadline or the upstream timeout after the request was sent), client_disconnect (the client hung up mid-response: 499) or retried (an abandoned or timed-out attempt after which the request went to the same upstream again). Tokens are the upstream's usage when it reported some, else estimates (prompt bytes/4, the relayed events' text at about 4 bytes a token); usd is list price, 0 on local upstreams. A 499 or broken response's own attempt is in cost_usd as well; the others are spend cost_usd doesn't show (0 / absent on rows before migration 8). cache_write_tokens is the share of tokens_in the provider reported writing to its prompt cache (billed at the price's cache_write, else input); cache_break is idle, model_change, prompt_shrank, concurrent or unknown when the request, in a session, read under half of the prefix it shares with the session's previous request on the same served model, "" otherwise (both 0 / "" on rows logged before migration 7). cache_saved_usd is what the prompt cache saved on the request at list price, priced when listed from the current table (as its base model when it has one): cached × (input − cached_input) − written × (cache_write − input), negative when a write cost more than it saved, 0 for local and unpriced models (also on the rows of GET /admin/v1/sessions/{id}). profile is the route profile the request was routed under, "" for none. variant is the experiment variant of a request to an experiment alias (route is then the alias), "" otherwise. attempt_log is why the request fell back: each target that did not serve as upstream/model:<reason> (the x-venom-fallback codes), then the one that did as upstream/model:ok, joined by → (openai/gpt-5.4-nano:429_quota→qwen/qwen3.8-flash:ok); "" when the first target tried served (and on rows logged before migration 5). Every admin read flushes the log writer first. &flush=1 also waits (up to 5 s) for requests that started but have not been logged yet: a client can read its last byte before the handler logs the request. &key=NAME scopes that wait to one client key (the rows are not filtered). The reply then carries x-venom-log-flushed: true, or false when the wait timed out
  • GET /admin/v1/sessions?limit=50 (max 500) → per-session aggregates over the request log, most recently active first: [{session_id, first_ts, last_ts, requests, errors, tokens_in, tokens_out, cached_tokens, cache_hit_ratio, local_tokens, cloud_tokens, cost_usd, models:[{upstream, model, local, requests, tokens_in, tokens_out, cached_tokens, cost_usd}], parent_session_id, subagents:{sessions, requests, errors, tokens_in, tokens_out, cost_usd}, variant, cache_write_tokens, cache_breaks}]. cache_write_tokens sums the session's requests' cache writes; cache_breaks counts its requests logged with a cache_break. variant is the session's experiment variant ("" when none of its requests went to an experiment alias). A session is every logged request that carried the same x-venom-session; requests without the header belong to none. A request may also carry x-venom-parent-session (venom's subagents send their parent's session id): it is logged as parent_session_id (ignored without x-venom-session, or when it names the session itself) and changes nothing in routing, except that on an experiment alias the subagent gets its parent's variant. Both headers are at most 256 bytes; a longer one is refused with 400 invalid_request_error, code invalid_session. x-venom-experiment: <id>; arm=<a|b> names a Venom Cloud gear experiment's arm (HarnessOptions.experiment, Harness.setExperiment; venom sets it when a session joins one, docs/cloud-mesh.md §10.14), sent wherever x-venom-gear goes. The orchestrator ignores it and never passes it upstream. x-venom-gear: <name> names the gear a request is made under: the harness sends the engaged gear's name (HarnessOptions.gear, Harness.setGear; venom's /gear sets it) on turns, summaries and side requests, and subagents take their parent's. The orchestrator ignores it; Venom Cloud counts the spend toward the org's gear budgets (docs/cloud-mesh.md §10.2) and answers x-venom-spend-warning (a budget past its warning share) and x-venom-spend-downgrade (a budget at its limit switched the model), which the harness turns into one notice line per budget and window, and 429 spend_limit_reached at a limit, which it explains (switch gears with /gear) without retrying. None of it enters the model context. x-venom-cache-ttl is the client's cache-lifetime intent on /v1 chat and Responses requests: long (venom's interactive TUI sends it; a person pauses between turns) asks a model with capabilities.cache_retention for that longer lifetime from the first request; short never does; absent, the session asks once one of its requests came more than 5 minutes after the previous one, and keeps asking for the rest of the session. The field is set per upstream kind (OpenAI prompt_cache_retention, Kimi on kind generic prompt_cache_options) and never overwrites a client's own. An answer to a request it was asked for carries x-venom-cache-retention: <value> (24h, 30m), which venom's keep-warm pings take as the cache's TTL. Unknown values count as absent. Not in the mesh dispatch allowlist: a request relayed by Venom Cloud falls back to the gap rule. parent_session_id is the session that started this one ("" for most). subagents sums every request of the sessions that name this one as parent; the other figures cover the session's own requests only, so its spend with its subagents is cost_usd + subagents.cost_usd. models lists what served the session (costliest first); a request no upstream served counts in the totals only. local_tokens / cloud_tokens are prompt + completion tokens on local / cloud upstreams. Figures cover the rows still in the log (request_log_retention prunes the oldest).
  • GET /admin/v1/sessions/{id}?limit=1000[&subagent_requests=1] (max 5000) → {session: {...as above}, requests: [...request rows, oldest first], subagents: [...sessions as above], outcome: {session_id, success, score, reported_at} | null, subagent_requests?: [...request rows]} (the newest limit requests when there are more; subagents are the sessions this one started, most recently active first, at most 500; outcome is the task outcome reported for it, below); 404 when no logged request carries the id. subagent_requests=1 adds every request of those subagent sessions, oldest first across them, the newest limit when there are more (absent without the parameter, or when there are none): the console's session trace
  • POST /admin/v1/sessions/{id}/outcome {success: bool, score?: number} → {session_id, success, score, reported_at, experiment, variant}. Records the session's task outcome (a later report replaces it; score is any finite number, null when left out), for the experiment's success rate. The id is 1–256 printable characters; the session need not be logged yet, and experiment / variant say what its requests were assigned ("" until the log has one of its experiment requests). 400 for a body without success, with unknown fields or over 4 KiB. venom's eval runner reports each trial's pass/fail here; a CI job can report its own verdict the same way
  • GET /admin/v1/shadow?days=7&route=<alias>&limit=50 → {days, routes: [R], samples: [S]}, shadow routing (docs/configuration.md → Shadow routing). R = {route, configured, config?, capped_today, counts: {sampled, capped, dropped_queue, dropped_capture, skipped, completed, failed}, candidates: [{route, candidate, samples, compared, agreed, errors, agreement, by_check, primary_p50_ms, shadow_p50_ms, primary_ttft_p50_ms, shadow_ttft_p50_ms, primary_cost_usd, shadow_cost_usd, shadow_cost_today_usd}]}: every configured route plus any with samples in the last days (1–30); counts since the process started. S = {ts, route, role, primary_model, candidate, candidate_model, request_id, shadow_request_id, check_kind, agree, score, detail, error, primary_ttft_ms, primary_ms, shadow_ttft_ms, shadow_ms, primary_cost_usd, shadow_cost_usd}, newest first (limit 0–500); agree and score are null for a failed replay. Replays' request-log rows carry source: "shadow"
  • GET /admin/v1/experiments → [{alias, variants: [{name, target, weight}]}], the configured experiment aliases
  • GET /admin/v1/experiments/{alias}?since=<RFC 3339> → {alias, configured, since, variants: [V]} with V = {name, target, weight, configured, sessions, subagent_sessions, requests, errors, error_rate, tokens_in, tokens_out, cached_tokens, cost_usd, latency_ms: {p50, p95, n}, ttft_ms: {p50, p95, n}, outcomes: {sessions, successes, success_rate, ci95_low, ci95_high, scored, mean_score}}. Over the retained request log's rows for the alias (at or after since when given; "" = all), per variant: sessions are top-level sessions, subagent_sessions subagent sessions with requests to the alias; errors are requests with status 0 or ≥ 400; latency percentiles (nearest rank) are over successful requests, TTFT over successful streams, n the sample size. outcomes joins the reported outcomes to the sessions of the variant (a session counts for the variant of its first request to the alias): success_rate = successes / sessions with its 95% Wilson score interval [ci95_low, ci95_high] [R190], mean_score over outcomes with a score; null without outcomes. Configured variants are listed (in config order) even before they have traffic; a variant a reload removed is listed after them, configured: false, while the log has its rows. 404 when the alias is neither a configured experiment nor in the log as one; 400 for a bad since
  • GET /admin/v1/keys → [{id, name, prefix, created_at, rpm_limit, tpm_limit, allowed_models, team, profile, budget: {daily_usd, daily_soft_usd, monthly_usd, monthly_soft_usd}, spend: {daily: P, monthly: P}}] with P = {spent_usd, reserved_usd, soft_usd, hard_usd, state: "ok"|"soft"|"exceeded"|"none", resets_at} (the current UTC day and month; reserved_usd is pre-charged by requests in flight; none = no limit for the period). prefix is the first 8 characters of a minted key (vk- + 5 of its 32 random characters), enough to tell keys apart. The bootstrap key (id: "env", from VENOM_API_KEY) has prefix: "": it is the operator's own string, possibly short or guessable, so no character of it is listed. It has no stored record, so it takes no team or budget (its spend is still tracked)
  • POST /admin/v1/keys {name, rpm_limit?, tpm_limit?, allowed_models?, team?, budget?, profile?} → {...key, key} (plaintext shown once). budget is {daily_usd?, daily_soft_usd?, monthly_usd?, monthly_soft_usd?} in USD, each 0–1e9, 0 = none; a soft limit left out is 80% of that period's hard limit, and may not exceed it. team is a label of at most 64 bytes grouping keys in the spend views. profile is the key's default route profile (used when a request sends no x-venom-profile); it must name a configured profile (400 otherwise), "" = none
  • PATCH /admin/v1/keys/{id} {team?, budget?: {...} | null, profile?, rpm_limit?, tpm_limit?} → the key as listed. profile: "" clears the default profile; a limit of 0 removes it (negative: 400). A changed limit applies to the next request and keeps the bucket's balance (no fresh minute). A field left out is unchanged; budget replaces the whole budget, null clears it. The key's spend so far is kept, so a lower limit applies to the next request. 400 for the bootstrap key, 404 for an unknown id
  • DELETE /admin/v1/keys/{id}
  • GET /admin/v1/spend?from=&to=&key=&team=&sessions=20 → the spend ledger for UTC days from..to (YYYY-MM-DD, inclusive; default the last 30 days; at most 366): {from, to, total: T, by_key: [{key_id, key_name, team, deleted, ...T}], by_team: [{team, keys, ...T}], by_model: [{model, base_model, ...T}], by_base_model: [{model, ...T}], by_day: [{day, ...T}], top_sessions: [{...session as in /sessions, total_cost_usd}]} with T = {requests, tokens_in, tokens_out, cost_usd}. by_model is the served upstream/model (the requested model for a request nothing served), with the base_model the current config prices it as; by_base_model sums it per priced-as model (the base model, else the served id without its upstream); by_day lists every day of the range. Teams are the keys' current teams ("" = none; a revoked key's spend stays, deleted: true, under ""); key_id: "" is spend without a key (open mode). key=<id> and team=<label> (team= alone = keys without a team) narrow every list. top_sessions are the costliest sessions active in the range, ranked by total_cost_usd = their own cost plus their subagents'; a subagent whose parent is also in the range is counted under it, not listed. They come from the request log, so they are limited to what request_log_retention keeps; the other lists come from the ledger, which is never pruned (one row per day, key and model)
  • GET /admin/v1/usage/by-role?since=7d&until= → spend per lane (VEN-13), grouped from the request log (so within request_log_retention): {object: "venom.usage.by_role", since, until?, requests, cost_usd, roles: [{role, requests, errors, tokens_in, tokens_out, cached_tokens, cost_usd}]}, costliest first. role is the x-venom-role the request named; a request that named none, or an unknown one, is main, so the lanes add up to the window and the total is the same as without the grouping. since is an RFC3339 time, a UTC day or a duration back (7d, 36h, 90m; default 7d; at most 366 days); until an RFC3339 time or a UTC day (default: no bound; 400 unless after since). errors counts status ≥ 400. venom cost --by role reads the same lanes from Venom Cloud's GET /v1/usage/summary.
  • GET /admin/v1/pricing → every configured model's effective price: {sources: ["custom","builtin"], table_models, models: [{id: "upstream/model", upstream, model, base_model, source, local, disabled, input, cached_input, output, blended, price_ref, routes: [alias]}], routes: [{alias, strategy, targets: [{id, source, blended}]}], unpriced: ["upstream/model"]}. Models are the route targets, the upstreams' static models lists and the upstream/model ids pricing: names; prices are USD per 1M tokens (null when unpriced), blended the 3:1 input:output price strategy: cheapest ranks by, source one of custom, base_model, builtin, local, unpriced. A route's targets are in its configured order (price order for cheapest). unpriced lists the enabled cloud models booked at $0 (docs/configuration.md → Pricing and cost-based routing).
  • GET /admin/v1/cache?days=7&sessions=10 → the prompt-cache report over the last days UTC days (1–366, today included; from the request log, so within request_log_retention): {since, totals: B, by_route: [B], by_model: [B], by_upstream: [B], by_day: [B], breaks: {reason: n}, top_sessions: [{...B, parent_session_id}]} with B = {key, requests, tokens_in, cached_tokens, cache_write_tokens, hit_ratio, cached_usd, uncached_usd, write_usd, saved_usd, breaks}. Rows are successful requests with a prompt and a serving upstream; cached_tokens is clamped to the prompt and cache_write_tokens to what is left of it; cached_usd + write_usd + uncached_usd is the prompt's list price (cached × cached_input, written × cache_write or input, the rest × input; 0 for local upstreams and unpriced models) and saved_usd = cached × (input − cached_input). breaks counts rows logged with a cache_break. by_model keys are the served upstream/model, by_day lists only days with rows, lists are by prompt tokens (days by date), and top_sessions (at most 100) are the sessions with logged breaks, most first, then by uncached + write $. The same figures as scripts/cache-report.py --json (make cache-report); a test runs both over one fixture log
  • POST /admin/v1/reload — re-read orchestrator.yaml and the price table. ${VAR} references expand from the process's environment as it started, so a changed .env value (a new provider key, say) needs a restart, not a reload
  • Config history (server/confighist.go, internal/confighist): every orchestrator.yaml the process applies (at start, on any reload: this endpoint, SIGHUP, a local-model selection; on a rollback) is stored in SQLite (config_versions, migration 10) as the file's bytes before ${VAR} expansion, with its SHA-256; a config equal to the newest row is not stored again, and the newest 50 are kept.
    • GET /admin/v1/config/versions → {versions: [{id, applied_at, sha256, source, bytes, current}], applied_sha256, file: {path, sha256, changed}, keep}, newest first. source is startup, reload or rollback:<id>; current marks the version the running config came from; file.changed says the file on disk differs from it (an edit not reloaded, or a reload that failed)
    • GET /admin/v1/config/diff?from=<id|file>&to=<id|file> (defaults: the newest version and the one before) → {from, to, diff: {upstreams, routes, profiles, other, changes}}; file is the file as it is now. Each section lists {name, op: added|removed|changed, value?, changes?: [{path, from?, to?}]}: upstreams by name, routes by alias, profiles by name, other one entry per remaining top-level key. Mappings diff per field (dotted path), lists whole. Secrets are redacted: ${VAR} references stay as written (the env value is never read), a literal header value or a literal under a key like key/token/secret/password becomes [redacted <6 hex of its SHA-256>] (a change still shows), a URL's password and query values become redacted. 404 unknown id, 400 a bad ref, 422 a side that isn't YAML
    • POST /admin/v1/config/versions/{id}/rollback {overwrite_sha256?} → writes the version to the config file (temp file in its directory + rename, the mode kept, a symlink's target written) and reloads it the ordinary way: {status: "rolled_back", rolled_back_to, version, upstreams, routes} (version is the new history row, source rollback:<id>), or {status: "unchanged"} when it is what runs already. 409 config_changed_on_disk {disk_sha256, applied_sha256} when the file isn't what was last applied, unless overwrite_sha256 equals disk_sha256 (the caller saw that content and replaces it); 422 invalid_version when the version no longer loads (checked before writing); 500 config_not_writable (e.g. compose's :ro config mount); 422 reload_failed puts the previous file back
  • Providers (server/providers.go, internal/providers, ADR 0045): set up provider keys from the console instead of editing .env. Every value is an environment variable the config already reads (api_key_env, ${X_BASE_URL:-…}, AWS_*); the config never holds a key. Values set here live in $VENOM_DATA_DIR/provider-keys.json ({"env": {VAR: value}}, 0600 in a 0700 directory, outside the repo) and are put in the process environment at start, before the first config load, unless the variable is already set (.env, the shell, compose): that value wins and is read-only here. A key is never returned or logged: responses say whether it is set, where from and its last four characters (none for a value under 12 characters).
    • GET /admin/v1/providers → {providers: [P], store: {path, persisted}, config: {path, writable}}. P = {id, name, section: cloud|local|voice, kind, upstream, blurb, key_help?, docs_url?, models_help?, verified?, shares?, fields: [F], configured, conflict?, status: {state, reason?, models, last_error?}}. A provider is listed only when this build has its kind (config.KnownKind); a speech card on a chat kind (MiniMax T2A, Qwen-TTS) only when the kind has a native speech adapter; a kind registered by an adapter with no template (a local TTS server added later) gets a generic card. shares names the provider whose upstream and key serve the card (no fields). configured: the config file has an upstream named upstream; conflict: that upstream has another kind. state is not_configured, disabled (reason, e.g. X is not set), checking, healthy or unhealthy. F = {id, label, type: secret|url|select|text|models, env?, placeholder?, hint?, pattern?, options?: [{value, label}], default?, optional?, set, source?: env|stored|config|default, value? (non-secret only), last4? (secret only), read_only, note?}. env is the variable the configured upstream reads (its own api_key_env wins over the template's); a base_url written out literally in the file has source: config, read-only. Field ids: api_key, base_url (a URL or a region select whose values are base URLs), aws_access_key_id, aws_secret_access_key, aws_session_token, aws_region, models
    • PUT /admin/v1/providers/{id} {values?: {field: value}, models?: [string]} → saves the values (a secret sent empty is unchanged; a key that doesn't match pattern is saved with a warning) and, when the config has no upstream for the provider, appends its block (the item's indentation matched, every other byte kept, api_key_env / ${X_BASE_URL:-default} only) and reloads, recorded in the config history as providers:<id>. models only when adding (default the template's). → {status: saved|unchanged, provider: P, config: {status: added|unchanged|manual, path, block?}, warnings: [string]}. A read-only config file (compose's :ro mount) keeps the values and answers status: manual with the block to add by hand. Errors {error, code}: 400 unknown_field, invalid_value, missing_value, shared_provider; 404 unknown_provider; 409 env_read_only (the variable comes from .env), field_fixed (written out in the config, or models of a configured upstream), field_unused (the upstream has no api_key_env), kind_conflict, config_not_editable (an inline upstreams: [...]), key_required (a typed-in url field changes while a key is set: the key must be sent again, so a stored key can't be pointed at another host; also on the test endpoint); 422 invalid_config (the new block doesn't load: nothing written), reload_failed (the previous file and values are put back)
    • POST /admin/v1/providers/{id}/test {values?, models?} → runs that upstream's own health check once (GET {base}/models with the kind's auth, or the kind's registered check) with values over the current ones, nothing saved: {ok, checked, latency_ms?, model_count?, models? (first 50), error?, message?, warnings}. AWS kinds and voice kinds with no check answer ok: true, checked: false. Errors never contain a key. The only outbound call, made only on demand
    • DELETE /admin/v1/providers/{id}/key → removes the provider's stored secrets and reloads; the upstream stays in the config, disabled. 409 env_read_only when the key comes from .env, 404 no_stored_key
    • 404 no_store on all four when the server runs without a key store (an embedder that didn't set Server.Providers)
  • GET /admin/v1/debug/pprof/ — the Go profiler (net/http/pprof): profile?seconds=N (CPU), heap, allocs, goroutine, block?seconds=N, mutex?seconds=N, trace?seconds=N. Block and mutex sampling are switched on only for the duration of a ?seconds= profile. Example: curl -H "Authorization: Bearer $VENOM_ADMIN_TOKEN" -o cpu.pb.gz 'http://127.0.0.1:8787/admin/v1/debug/pprof/profile?seconds=10' then go tool pprof -top cpu.pb.gz (go tool pprof can't send the header itself)

Local models (/admin/v1/local/*, internal/localmodels): the daemon's own models, so a user never has to drive Ollama or vLLM by hand. Off the /v1 path; outbound calls go only to the configured local upstreams and to llmfit. Errors are {error, code} with a stable code.

  • GET /admin/v1/local → {hardware, llmfit, backends, selection, strain, pulls, mesh}:
    • hardware: {source:"llmfit", gpu_name, gpu_count, gpu_vram_gb, unified_memory, total_ram_gb, available_ram_gb, cpu_name, cpu_cores, backend}, or null without llmfit; llmfit: {available, via:"exec"|"url"|"", error}.
    • backends: one per enabled upstream of kind ollama, vllm, llamacpp, sglang: {upstream, kind, base_url, reachable, version, managed, error, installed:[{model, size_bytes, parameter_size, quantization, family, modified_at}], loaded:[{model, size_bytes, vram_bytes, context_length, expires_at}], served:[string], metrics:{running, waiting, kv_cache_pct}|null}. Every field is always present: installed/loaded are Ollama's (/api/tags, /api/ps), served vLLM's /v1/models (llama.cpp / SGLang: the health check's list), metrics vLLM's /metrics. managed marks the Ollama the actions below use (the first enabled one).
    • selection: {default, fast, source:"env"|"console"}: the models venom-default and venom-fast route to on the managed Ollama; console when a choice made here is in force.
    • strain: per backend {upstream, model, level, inflight, max_inflight, utilization, latency_ewma_ms, healthy, reasons:[string]}, level the first that applies: down (no answer, or the last health check failed), strained (vLLM requests waiting or KV cache ≥ 90%; an Ollama model partly in system RAM, vram_bytes < size_bytes; every max_inflight slot taken), busy (half the slots or more), ok (something in flight), idle.
    • pulls: [{model, status, completed, total, percent, error, started_at, finished_at, select}], running pulls and those finished in the last 10 minutes; status is Ollama's latest line while running, then success or error.
    • mesh: {enabled, cloud_url, name, preflight?, live?} (cloud_url/name present when VENOM_CLOUD_URL is set; enabled when mesh mode runs). live (only while mesh mode runs, in-memory, no network call) is {daemon_id, state:"connecting"|"connected"|"reconnecting"|"draining"|"stopped", error?, last_heartbeat_at?, lease_expires_at?, rtt_ms|null, channel_open, inflight, slots, jobs_done}: connected while the last heartbeat's lease runs, reconnecting once it ran out, stopped (with error) after a revoke or a protocol refusal; jobs_done counts dispatched jobs finished since start. The console's overview reads it.
  • GET /admin/v1/local/catalog[?refresh=1] → {hardware, llmfit, context, recommended:{default, fast}, models:[{id, backend:"ollama"|"vllm", name, hf_name, agent_ready, size_gb, memory_gb, fit:"perfect"|"good"|"marginal"|"too_tight"|"unknown", run_mode:"gpu"|"cpu_offload"|"moe_offload"|"cpu"|"", est_tps, tools, vision, note, installed, selected:"default"|"fast"|"", pullable, hint, chat}]}. context is the managed Ollama's context_window (OLLAMA_CONTEXT_LENGTH, default 32768). Rows: the curated agent-ready Ollama tags (agent_ready: true; memory_gb = the tag's weights plus its KV cache at context, fitted against llmfit's VRAM/RAM); scripts/model.sh's vLLM presets (agent_ready: true, sized for VRAM alone); llmfit's other tool-using Ollama models that fit wholly on the GPU (≤ 20; a model with no Ollama tag but an Unsloth GGUF in llmfit's gguf_sources is listed as hf.co/unsloth/<repo>:<quant>, llmfit's best_quant when it is a GGUF quant, else Q4_K_M; memory_gb at least the download + 10%, since llmfit counts only an offloaded MoE's GPU part); up to five llmfit vLLM picks from the makers' own repos (Qwen/, openai/, mistralai/, google/, meta-llama/, deepseek-ai/, zai-org/); every vLLM row has pullable: false, hint: "make model BACKEND=vllm MODEL=<repo>" and launch (true when POST /admin/v1/local/serve can start it: a known tool-call parser, not too_tight, and the catalog's vllm_launch.available); then installed tags in neither. chat: false marks an installed embedding model (a *bert family or "embed" in the name): not selectable. Order: selected, agent-ready models that fit (biggest first), other models that fit, the rest, embedding models last. recommended (Ollama rows only): the largest agent-ready model that fits perfect or good, and the smallest that fits perfect; qwen3:8b / qwen3:4b without llmfit. llmfit is read once per 30 minutes (refresh=1 re-reads).
  • GET /admin/v1/local/suggest[?days=7&gpu_usd_per_hour=X] → {object: "venom.local.suggest", days, window_days, gpu_usd_per_hour|null, local: {installed, fast|null, default|null}, candidates: [{gear, role, model, requests, requests_per_week, avg_tokens_in, avg_tokens_out, tool_share, avg_latency_ms, cost_usd, cost_per_week, local_model|null, gpu_per_week_usd?, save_per_week_usd, fit: "good"|"maybe"|"poor", note?}], total_save_per_week_usd, note}: the local-model autopilot's read (venom models local suggest). The successful (status < 400) cloud (local = 0) requests of the last days (1 to 30, default 7) whose role is classifier, summary, reflex or workers, grouped by gear, role and served model (at most 200 groups, costliest first; main turns, smart, vision and browser are left out: a small local model is not a stand-in for them). window_days is what the per-week figures scale from: days, or less when the window's first such row is younger (roles are logged from migration 15 on), at least 1. local_model is what would serve the role here: venom-fast's model for classifier, summary and reflex, venom-default's for workers, counted only when the managed Ollama has it installed (without a reachable Ollama, the alias's first healthy local target), else null. fit (localmodels.SuggestFit): good for classifier, summary or reflex averaging ≤ 8k tokens in and ≤ 1k out; maybe for workers, or up to 32k in; poor for larger, and for a classifier or summary with more than half its requests offering tools. save_per_week_usd is cost_per_week (local tokens are $0) minus, with gpu_usd_per_hour (0 to 1000), gpu_per_week_usd = requests a week × the rows' average latency / 3.6e6 × $/h (the cloud's latency as a rough stand-in for the local model's); without it every candidate's note says GPU time and electricity are not counted. Candidates are sorted by save_per_week_usd, highest first; total_save_per_week_usd sums the good and maybe ones that save. Read only: nothing is applied. 400 invalid_days, 400 invalid_gpu_usd_per_hour.
  • POST /admin/v1/local/pull {model, select?: "default"|"fast"} → 202 {pull}: starts (or joins) a background POST /api/pull on the managed Ollama; it outlives the request and ends with the server. With select, the model is selected for that tier when the pull succeeds. 400 invalid_model (Ollama's name charset, hf.co/<user>/<repo>:<quant> allowed) / invalid_tier, 409 no_ollama (no Ollama upstream), 503 ollama_unreachable.
  • POST /admin/v1/local/select {model, tier:"default"|"fast"} → 200 {selection, warning?}: sets VENOM_OLLAMA_MODEL / VENOM_OLLAMA_FAST_MODEL in the process and reloads the config exactly as POST /admin/v1/reload does (a failed reload restores the variable, 500 reload_failed), then persists $VENOM_DATA_DIR/local-models.json (0600, {"default":{"model","env"},"fast":{…}}, env = the variable when the process started). At start, before the first config load, each entry is applied unless the variable now differs from its env (make model or .env changed it since: that choice wins and the entry is dropped). 409 not_installed when Ollama doesn't have the tag (pull with select instead), 409 not_chat for an embedding model; warning when the config doesn't route the tier's alias through its variable. After a select (and a pull), the managed Ollama's health check runs at once, so routing sees the model without waiting for health_interval.
  • POST /admin/v1/local/setup (no body or {}) → 202 {recommended, pulls, selection} when it started pulls, 200 with pulls: [] when both recommended models are installed and selected: the one-click path, pulling what is missing (selected on success) and selecting what is installed.
  • POST /admin/v1/local/serve {model} → 202 {serve}: one-click vLLM. The orchestrator starts vLLM itself as one Docker container, venom-vllm (compose's pinned image, VENOM_VLLM_IMAGE overrides; flags and parsers as make model BACKEND=vllm; -p 127.0.0.1:<vllm upstream port>:8000; weights in compose's venom_hf-cache volume; --restart unless-stopped), after unloading the managed Ollama's loaded models and removing its own previous container. It follows the start in the background (GET /admin/v1/local → serve: {model, state: starting|ready|error, message, started_at, finished_at}, message the container's last log line), and once vLLM lists the model sets VLLM_MODEL and reloads, persisted as vllm in local-models.json (same precedence rule). Available only when the orchestrator runs on Linux with docker on its PATH (VENOM_DOCKER_BIN), Docker has the NVIDIA runtime, and the vllm upstream's base_url is plain http on loopback (GET /admin/v1/local/catalog → vllm_launch: {available, reason}): so on the desktop daemon and make dev-orchestrator, not in the compose container. 400 invalid_model (a Hugging Face owner/name) / no_parser, 409 vllm_launch_unavailable, 409 vllm_foreign (a vLLM it didn't start answers on the port: left alone). Selecting an Ollama model (select, setup, a pull with select) stops this container: one GPU, one engine.
  • DELETE /admin/v1/local/serve → 204: removes venom-vllm; 404 not_running.
  • DELETE /admin/v1/local/models/{model...} → 204; 409 selected for a selected model, 409 pulling, 404 not_installed.
  • POST /admin/v1/local/unload {model} → 204 (Ollama keep_alive: 0: frees its VRAM now).

Daemon (/admin/v1/mesh*, /admin/v1/daemon*; internal/meshctl, internal/daemon): what the desktop daemon's dashboard drives (docs/daemon.md). Errors are {"error": {message, type, code}}. Nothing here dials out until the user connects (mesh) or turns on or asks for an update (auto-update).

  • GET /admin/v1/mesh → {source: "env"|"dashboard"|"none", state: "off"|"connecting"|"connected"| "revoked"|"error", cloud_url, name, daemon_id, team_id, region, enrolled_at, error, default_cloud_url: "https://cloud.usevenom.dev", can_change, paused, jobs, tokens_per_s}. paused (below), jobs (dispatched jobs running now) and tokens_per_s (output tokens per second of the requests finished in the last minute, null when none) read memory only; the macOS menu bar app polls them. source: "env" (mesh mode set by VENOM_CLOUD_URL) is read-only here: can_change is false and connect/disconnect answer 409 mesh_managed_by_env.
  • POST /admin/v1/mesh/connect {join_token: "vjt_…", cloud_url?, name?, region?} → 200 status: enrolls (docs/cloud-mesh.md §6.1), saves $VENOM_DATA_DIR/mesh-settings.json (cloud URL, name, region; never the token) next to the credential in mesh/, and starts mesh mode without a restart; a restart reads the settings file like VENOM_CLOUD_URL/VENOM_MESH_NAME/VENOM_MESH_GEO. cloud_url defaults to https://cloud.usevenom.dev, name to the hostname. 400 invalid_join_token|invalid_cloud_url|invalid_name|invalid_region, 409 mesh_already_connected, 502 mesh_join_failed (the cloud's reason, e.g. an expired token).
  • POST /admin/v1/mesh/disconnect → 200 status: drains running jobs, deregisters, removes the credential and the settings file (and a pause).
  • POST /admin/v1/mesh/pause / POST /admin/v1/mesh/resume → 200 status: the daemon stays connected and sends its draining heartbeat (the cloud stops dispatching; running jobs finish) until resumed. Kept across restarts as $VENOM_DATA_DIR/mesh-paused; the auto-update's drain and its resume do not end a pause. 409 mesh_off with no connection configured (venom-daemon pause|resume).
  • GET /admin/v1/heal?limit=50 (1 to 200) → {state: "ok"|"healing"|"degraded", failing: [{name, kind, state: "healing"|"degraded"|"attention", attempts, since, next_try?, error?}], recovered_total, repairs_total, events: [{at, kind, target, action?, outcome, attempt?, detail?}]} (newest first): the self-heal supervisor (internal/heal, docs/daemon.md → Self-heal). 404 not_available without one. Read-only.
  • GET /admin/v1/computers → {enabled, state: "off"|"enabling"|"on"|"error"|"unsupported", source?: "env"|"setting", can_enable, message?, checks: [{id, label, ok, detail?, fix?}], steps: [{id, label, state, detail?, progress?}], image: {ref, present, way?}, capacity?} (VEN-403, Venom Computers one-click setup, docs/public/computers/enable.md). POST /admin/v1/computers/enable runs the checks, gets the image, saves computers/setting.json in the data dir and starts the manager live: 202 while it works, 200 once on, idempotent; 409 unsupported on a host that cannot run Computers (macOS, a container); any other failed check ends in state error with its fix sentence. POST /admin/v1/computers/disable hibernates seats, stops and saves the setting off (409 env_override while VENOM_COMPUTERS=on). 404 not_available without the feature. The same switch is the mesh computer job ops enable and disable (seat_id the nil uuid).
  • GET /admin/v1/daemon → {managed, version, os, autostart: {supported, enabled}, ollama: {installed, path}, ollama_download_url}. managed is true when venom-daemon started the orchestrator (VENOM_DAEMON_LAUNCHER); Ollama's installed is a file check (PATH and its usual install paths); whether it runs is the ollama backend's reachable in GET /admin/v1/local.
  • PUT /admin/v1/daemon/autostart {enabled} → the daemon status; writes or removes the per-user login item (a LaunchAgent, an XDG autostart entry, a Startup-folder script). 404 not_managed outside the launcher.
  • GET /admin/v1/daemon/projects → {folders: [{path, exists}], max, serve: {running, url?, error?}, managed, workspaces?: {root, items: [{id, name, kind, path, origin, created_by?, git_url?, created_at}], scratch_ttl_hours}}: the folders the daemon's venom serve serves next to its workspaces folder (ADR 0081, docs/daemon.md → Projects; projects.json in the state dir) and the workspaces in it (docs/daemon.md → Workspaces). DELETE /admin/v1/daemon/projects?workspace=<id> deletes one and its files (404 workspace_not_found). POST /admin/v1/daemon/projects {path} adds one (absolute, or under the home folder for ~/… and relative paths; must be an existing folder, not a disk root; at most 50; 400 invalid_project) and DELETE /admin/v1/daemon/projects?path= removes one (404 project_not_found); both run the launcher's serve-reload (restart serve) and answer the list, with serve.error when serve did not come up. 404 not_managed outside the launcher.
  • Auto-update (internal/daemon/update.go, ADR 0031; opt-in, docs/daemon.md → Auto-update). The daemon status carries it as update (managed daemons only), and:
    • GET /admin/v1/daemon/update → {supported, reason?, auto_update, state, current, latest?, available, last_check_at?, next_check_at?, error?, detail?, progress?: {done, total}, skipped_versions?, last_result?: {from, to, state, reason?, at}, source}. state is idle|checking|available|downloading|verifying|waiting_for_idle|draining|restarting|done|failed|rolled_back; supported: false (with reason) for a dev build, Windows, or an install not made by install-daemon.sh. No network.
    • PUT /admin/v1/daemon/update {auto_update} → the update status. Off by default; on, checks run every 6 h ±20% (the first within 10 min), backing off with full jitter on failure.
    • POST /admin/v1/daemon/update/check → the update status after one GET of <releases>/latest/download/SHA256SUMS (VENOM_DAEMON_UPDATE_URL, default https://github.com/elishahbiz/venom-releases/releases); 502 check_failed.
    • POST /admin/v1/daemon/update/apply → 202 the update status; the install runs in the background (download, SHA256 against the version's SHA256SUMS, unpack beside the running release, an idle window of 30 s with no /v1 request or mesh job in flight, at most 30 min, a draining mesh heartbeat, then the new launcher's venom-daemon upgrade swaps and restarts, or rolls back). 409 up_to_date|update_busy|not_supported.
    • All four answer 404 not_managed outside the launcher. Files: data/daemon-update.json (switch, last check, versions skipped after a rollback) and data/daemon-upgrade.json (the swap's result), both 0600.

Cost. cost_usd is list price from the bundled, offline price table (config/prices.yaml: USD per 1M input, cached_input and output tokens, with the date checked and the source URL per model; never fetched at runtime). Per request: (prompt − cached) × input + cached × cached_input + completion × output, matched on the model the upstream served (exact, else minus a dated snapshot suffix). A local upstream (kind ollama, vllm, sglang, llamacpp, or a base URL whose host is loopback, private, link-local, *.local, *.internal or a single label such as a compose service) costs 0 and is marked local: true, unless the upstream's config sets local: false (a paid gateway on a private address, such as a LiteLLM proxy: billed, and counted against budgets) or local: true (a self-hosted server on a public name); a cloud model missing from the table also costs 0 (logged once at warn, listed by GET /admin/v1/pricing). pricing: in orchestrator.yaml sets custom prices and base models (a deployment named my-gpt priced as gpt-5.4) and the lookup order over custom and the table; a served model with a base model is answered with x-venom-base-model (docs/configuration.md → Pricing and cost-based routing). It is computed once per request after the response is relayed, never on the streaming path. The same table ships with the harness for the CLI's local estimate (packages/harness/src/cost/prices.ts, kept in sync by a test).

Model catalog: GET /api/models/providers/{id} (no key, not under /v1) serves <VENOM_MODEL_CATALOG>/<id>.json, a {"models":[...]} shard in the format pi's remote catalog client reads, with a strong ETag and Last-Modified (If-None-Match → 304), types= ignored; an unknown or unsafe id, or no directory, is 404. Static files, no per-request allocation beyond the file handle. The CLI does not read it yet: pi has no setting for the catalog URL (ADR 0042).

Ops: GET /healthz (liveness, 200 {status:"ok"}), GET /readyz (200 with ≥1 healthy upstream, else 503 {status:"no healthy upstream", healthy_upstreams:0, checking_upstreams:0}, or status:"checking upstreams" with checking_upstreams > 0 while enabled upstreams still await their first probe, right after a start or reload: venom doctor waits up to 6 s for those); an upstream counts as unhealthy until its first probe passes; a probe that only timed out keeps a healthy upstream healthy when live traffic got a 2xx from it since the previous probe, and otherwise takes two in a row, while an error status or a refused connection still counts at once), GET /metrics (Prometheus text format 0.0.4, passes promtool check metrics):

metric type labels
venom_requests_total counter model, upstream, code (0 = no upstream reached, 499 = client closed)
venom_request_duration_seconds histogram model (successful requests)
venom_ttft_seconds histogram model (successful streams)
venom_tokens_total counter model, type = prompt | completion | cached | cache_write
venom_cache_breaks_total counter model (served upstream/model), reason = idle | model_change | prompt_shrank | concurrent | unknown (see /admin/v1/requests cache_break)
venom_cache_hit_ratio gauge — (cached / prompt tokens since start)
venom_cost_anomaly gauge scope = key | session, metric = spend | cache_breaks (open anomalies, /admin/v1/anomalies)
venom_cost_anomalies_total counter scope, metric (anomalies opened)
venom_response_cache_hits_total counter kind (side kind), how = hit (stored) | shared (an identical request in flight); billed 0
venom_response_cache_stores_total counter kind (side-request responses stored)
venom_dedup_total counter how = joined (shared an identical request's upstream call) | replayed (a stored Idempotency-Key response) | lagged (cut loose: fell max_lag_bytes behind) | alone (too late to join, went upstream itself) | mismatch (an Idempotency-Key reused with another body, 422); billed 0
venom_tool_call_repairs_total counter upstream (served), kind = wrapper | single_quote | python_literal | control_char | trailing_comma | unterminated | unrepaired (responses whose tool-call arguments were mended; see services/go-orchestrator/AGENTS.md → Tool-call repair)
venom_upstream_shaping_queue_depth gauge upstream (requests waiting for the provider's learned rate limit; only upstreams whose shaper learned a limit)
venom_upstream_shaping_requests_total counter upstream, outcome = admitted (at once) | queued (after waiting) | cancelled (left the queue)
venom_upstream_shaping_rejected_total counter upstream, reason = queue_full | wait_exceeded (answered 429 here)
venom_upstream_shaping_wait_seconds histogram upstream (how long queued requests waited)
venom_upstream_requests_waiting, venom_upstream_kv_cache_usage gauge upstream (the engine's own queue and KV-cache use 0–1 from its /metrics; fresh scrapes only)
venom_cost_usd_total counter model (list-price USD from the price table; local upstreams add 0)
venom_wasted_tokens_total, venom_wasted_usd_total counter model, reason = abandoned | ttft_timeout | client_disconnect | retried (billed attempts whose output was discarded; see /admin/v1/requests waste)
venom_active_streams, venom_uptime_seconds gauge —
venom_upstream_healthy, venom_upstream_inflight gauge upstream
venom_upstream_circuit_state gauge upstream (0 closed, 1 open, 2 half_open)
venom_upstream_latency_ewma_seconds gauge upstream
venom_upstream_attempts_total counter upstream, attempt = first | retry (a retry is a later attempt of the same request on the same upstream)
venom_upstream_retries_skipped_total counter upstream, reason = budget (retry budget spent) | retry_after (the upstream asked for more than backoff_max; its answer went to the client) | client_retrying (x-stainless-retry-count > 0)
venom_upstream_retry_budget_available gauge upstream (retries the budget allows now)
venom_upstream_quota_exhausted gauge upstream, model (1 while a target sits out its quota cooldown; no series otherwise)
venom_route_target_available gauge route, upstream, model (1 = the upstream has the model or can't tell, 0 = routing skips the target)
venom_key_spend_usd gauge key (key name), key_id, period = daily | monthly (spend in the current UTC day / month)
venom_key_budget_usd gauge key, key_id, period, limit = soft | hard (only limits that are set)
venom_budget_soft_limit_crossed_total counter key, key_id, period (at most once per key and period window)
venom_budget_exceeded_total counter key, key_id, period (requests refused with 429 budget_exceeded)
venom_request_log_dropped_total counter —
venom_request_log_rows gauge — (rows in the request log: request_log_retention, plus up to a margin of 10%, at least 2,048, before pruning starts)
go_goroutines, go_heap_objects_bytes, go_memory_total_bytes, process_open_fds gauge — (process_open_fds on Unix only)
go_gc_cycles_total, process_cpu_seconds_total counter — (process_cpu_seconds_total on Unix only)

The four per-key series list a key with a budget (every series, zeros included), else a key with spend this month or a non-zero counter; an idle key without a budget adds none. key_id keeps two keys with the same name apart.

model is the requested alias or id (unknown names are recorded as (unknown), and a direct upstream/<model> the upstream neither lists nor has configured as a route target as upstream/(unknown)), so label cardinality is bounded by the config and the upstreams' model lists. The observability compose profile (Prometheus + Grafana, config in deploy/observability/) graphs these.