Your models, any provider, one endpoint, and a plan B.
Put your own GPUs and the cloud providers you choose behind one OpenAI-compatible endpoint. Every request is logged and budgeted, and when a provider fails, it moves to the next target your route allows.
The daemon and the venom CLI are free downloads.
What it looks like today
- Half your traffic goes to one provider, and an outage or a price change on their side stops your product.
- The GPUs you bought sit behind a different API from the providers, so every app picks one and stays there.
- Keys are pasted into each service, and nobody can say what a team spent or which model answered.
- Switching a model means a code change in every client.
What Venom adds
It doesn't replace vLLM, Ollama or your providers. It sits in front of them.
- The orchestratorOne OpenAI-compatible endpoint, streamed without buffering. Route aliases map to whatever serves them today, so a client never changes when a model does. Docs
- The routerKeeps a session on the backend that holds its prompt cache, skips targets that are down or out of quota, and tries several keys per provider by priority. Docs
- The daemonThe orchestrator and its dashboard on your own machine, in front of Ollama, vLLM or llama.cpp, dialling out to Venom Cloud. Docs
- Keys and budgetsA key per person, service or CI job, with its own rate limits and spend budget, checked before the request goes anywhere. Docs
- Logs and metricsEvery request with its key, model, target, tokens, cost and latency, a one-line query language over them, and Prometheus metrics with a Grafana dashboard. Docs
- A failover drillPick a provider, key or daemon to take down and see, without taking anything down, what fails over, to what, and at what price. Docs
One endpoint in front of everything
Clients call one URL. The endpoint picks a target in the order your route sets, keeps sessions where their cache is warm, and moves on before the first byte when a target fails.
Your clients
- /v1/chat/completions
- /v1/responses
- /v1/embeddings
One endpoint
Your GPUs
- vLLM
- SGLang
- llama.cpp
- Ollama
Providers on your keys
- OpenAI
- Z.ai
- Qwen
- TypeSafe
Public nodes you allowlist
What is true today
Each line links the page of the docs that shows it.
- Chat, completions, responses, embeddings and the model list, streamed as they are generated.docs/orchestrator/chat-and-responses
- A target that failed its health check, has an open circuit breaker or sits out a quota cooldown is skipped without an attempt.docs/orchestrator/routing
- Make an alias that tries the cheapest target first, choose it per key or request, and read back what each request cost.docs/cookbook/cost-routing
- Join a machine to your mesh with a token from the daemon's dashboard. It opens no inbound port.docs/mesh/connect-daemon
- Search, group and chart the request log in one line, in the console, in alerts and over the admin API.docs/cloud/console-log-query
- Spend per key group, project, gear, key or member, a monthly cap, and chargeback statements.docs/cloud/console-usage
- Scrape the orchestrator's /metrics from your own Prometheus, or run the provisioned Grafana dashboard.docs/deploy/observability
- Unplug the network and the venom CLI still runs against a local orchestrator and model.docs/cookbook/offline-ollama
Works with the code you have
Speaks the OpenAI API
- /v1/chat/completions
- /v1/responses
- /v1/embeddings
- /v1/completions
- /v1/models
Providers on your keys
- OpenAI
- Z.ai
- Qwen (DashScope)
- TypeSafe
Backends on your daemons
- vLLM
- SGLang
- llama.cpp
- Ollama
Change the base URL to https://cloud.usevenom.dev/v1, use a vck_ key, and keep your SDK.
Each request goes where its prompt cache is warm
The same router runs in the daemon, in front of your backends, and in Venom Cloud, across your daemons. It keeps a session on the GPU that holds its prompt and fails over before the first byte.
Run any model, without an account or telemetry
venom runs the model you choose, local or hosted, with no account, and it ships with telemetry and update checks off. Here is how that looks next to Claude Code and Qwen Code.
- venom CLI
- Any OpenAI-compatible endpoint: Ollama, vLLM, SGLang, llama.cpp, Venom Cloud or a provider
- Claude Code
- ~Claude models, through Anthropic or Amazon Bedrock, Google Cloud and Microsoft Foundry
- Qwen Code
- OpenAI, Anthropic, Gemini and Qwen APIs, any OpenAI-compatible endpoint, local models
- venom CLI
- None. Point it at a model and start
- Claude Code
- A Claude Pro, Max, Team, Enterprise or Console account, or a cloud provider
- Qwen Code
- A provider key, or a local model
- venom CLI
- Works, with a local model
- Claude Code
- –Docs list an internet connection as required
- Qwen Code
- Not checked
Usage telemetry by default1,6,7
- venom CLI
- Off. No telemetry and no update checks
- Claude Code
- On with the Claude API (no code or prompts), off on cloud providers; one variable turns it off
- Qwen Code
- On; one setting turns it off
| Capability | venom CLI | Claude Code | Qwen Code |
|---|---|---|---|
| Models1,2,3,4 | Any OpenAI-compatible endpoint: Ollama, vLLM, SGLang, llama.cpp, Venom Cloud or a provider | ~Claude models, through Anthropic or Amazon Bedrock, Google Cloud and Microsoft Foundry | OpenAI, Anthropic, Gemini and Qwen APIs, any OpenAI-compatible endpoint, local models |
| Account needed5,2,4 | None. Point it at a model and start | A Claude Pro, Max, Team, Enterprise or Console account, or a cloud provider | A provider key, or a local model |
| With the network unplugged1,2 | Works, with a local model | –Docs list an internet connection as required | Not checked |
| Usage telemetry by default1,6,7 | Off. No telemetry and no update checks | On with the Claude API (no code or prompts), off on cloud providers; one variable turns it off | On; one setting turns it off |
added per request, p50
Auth, alias resolution, quirks, usage accounting and session affinity on the path. Streaming adds +65 µs p50, and nothing is buffered.
make bench on an Apple M2 Pro with a local fake upstream.
cached on a new session's first request
Sessions share a small pool of OpenAI cache keys, so a new session finds the common prompt prefix already cached.
Live, eight new sessions with the same 5k-token prefix.
pass@1 on Venom's eval suite
42 tasks, 3 trials each: 95.2% passed all 3. The safety slice passed 18 of 18 with no permission-engine violations. 126 trials cost $1.23, with 74.4% of input tokens cached.
Self-authored tasks, gpt-5.4-mini through the orchestrator. Not a public benchmark.
The full comparison, with every speedup
Sources
- 1.Venom README: the CLI turns off telemetry, update checks and downloads, and the orchestrator calls only its configured upstreams. README.md:32-33.
- 2.Claude Code setup docs: requires a Pro, Max, Team, Enterprise or Console account, or a third-party provider (Amazon Bedrock, Google Cloud, Microsoft Foundry); "internet connection required"; native installs update in the background, package-manager installs don't by default. code.claude.com/docs/en/setup. Checked 2026-10-18.
- 3.Claude Code LLM gateway docs: Anthropic "doesn't support routing Claude Code to non-Claude models through any gateway." code.claude.com/docs/en/llm-gateway. Checked 2026-10-18.
- 4.Qwen Code README: originally based on Google Gemini CLI v0.8.2, developed independently since v0.1; Apache-2.0; supports OpenAI, Anthropic, Gemini and Qwen APIs and local models (Ollama, vLLM). github.com/QwenLM/qwen-code. Checked 2026-10-18.
- 5.The venom CLI, the orchestrator and the cloud are MIT licensed; the CLI needs no account. cli/package.json:7, docs/release.md:144.
- 6.Claude Code data-usage docs: metrics are on by default with the Claude API (they never include code, prompts or file paths), off by default on Bedrock, Google Cloud and Foundry; DISABLE_TELEMETRY or CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC turns them off. code.claude.com/docs/en/data-usage. Checked 2026-10-18.
- 7.Qwen Code settings docs: privacy.usageStatisticsEnabled defaults to true; general.enableAutoUpdate (update checks and installs on startup) defaults to true; approval modes plan, default, auto-edit, auto and yolo. qwenlm.github.io/qwen-code-docs/en/users/configuration/settings. Checked 2026-10-18.
- 8.make bench, Apple M2 Pro, a local fake upstream, a ~5 KB chat request with a tool; the added time includes the extra loopback hop. Re-runs on a loaded machine measured +59 to +63 µs. services/go-orchestrator/AGENTS.md "chat body with a tool; non-stream: one JSON response;".
- 9.OpenAI prompt_cache_key pooled across sessions, measured live: eight new sessions with the same 5k-token prefix. CHANGELOG.md "live, eight new sessions with the same".
- 10.Venom eval baseline, 2026-10-09: 42 self-authored tasks in 8 categories, k=3 (126 trials), venom-default routed to gpt-5.4-mini through a local orchestrator. Not a public benchmark and not a head-to-head with other agents. evals/results/2026-10-09T08-33-43Z-23be.json, evals/AGENTS.md "`apply_patch` and no `edit`. Results: pass@1".
- 11.Bounded-load session owner, simulated (npm run sim:bounded): 10 daemons × 4 slots, Poisson arrivals, 1,000 Zipf-popular sessions (s = 0.8 and 1.1), 50k requests. A simulation, not a production measurement. docs/cloud-mesh.md "(`bun run sim:bounded`, `sim/bounded-load.ts`). The setup".
- 12.Venom Cloud's route() wall-clock time on Node 24 for 10 eligible daemons (65 µs p50 for 100). docs/cloud-mesh.md "Only the `route()` timings are wall-clock measurements", docs/cloud-mesh.md "`route()` itself takes".
- 13.Load test against a fake vLLM upstream (300 ms TTFT, 100 tokens/s) on a shared M2 Pro; numbers are at 1,000 concurrent streams. services/go-orchestrator/AGENTS.md "tokens/s per stream,", services/go-orchestrator/AGENTS.md "**Streaming levels**".
Ask your request log a question in one line
Filter by model, key, status, latency, cost or daemon, then pipe into counts, percentiles, top lists or a time series. The console colours and completes the query as you type.
model:gpt-* status:5xx latency:>2s | p95(latency) by provider| provider | p95(latency) | bar | requests |
|---|---|---|---|
| openai | 3.42 s | 41 | |
| qwen | 2.87 s | 18 | |
| zai | 2.31 s | 7 |
Example output. Runs over your org's request log in the console.
More you can ask
key:ci-* cost:>$0.05 @todayexpensive calls from CI keys today
daemon:gpu-box ttft:<300msfast first tokens on one daemon
| top 5 model by sum(cost)the five costliest models
| timeseries 5m count by statusrequests per status, as a chart
A coding agent on your own models
The venom CLI works in your terminal on your real repository and speaks the same API. Point it at your mesh, and every call shows up in the log under your key's budget.
- Four permission modes, with shell commands parsed and judged part by part
- Subagents, plan mode and /undo checkpoints
- Claude Code projects, plugins, hooks and MCP servers load as they are
- With a local model it works offline
Your keys pass through at cost
Traffic on your own provider keys and your own daemons carries no markup. When Venom supplies the inference, on its own provider keys or the managed public mesh, you pay the provider's cost plus 20%.
- The orchestrator, the daemon and the venom CLI are free downloads
- Venom Cloud plans are a flat fee for the organisation, with unlimited collaborators
- Budgets and spend caps per key, key group and organisation
Questions people ask first
What is a daemon?
Does it work with the code I already have?
https://cloud.usevenom.dev/v1, use a vck_ key from the console, and keep your SDK.What happens when a provider goes down?
Can I run it without Venom Cloud?
Can I mix local and cloud models in one route?
Send your first request to your own hardware today
Sign up, connect a daemon or add a provider key, and point any OpenAI client at one URL.