Skip to content

Your models, any provider, one endpoint, and a plan B.

Put your own GPUs and the cloud providers you choose behind one OpenAI-compatible endpoint. Every request is logged and budgeted, and when a provider fails, it moves to the next target your route allows.

The daemon and the venom CLI are free downloads.

What it looks like today

  • Half your traffic goes to one provider, and an outage or a price change on their side stops your product.
  • The GPUs you bought sit behind a different API from the providers, so every app picks one and stays there.
  • Keys are pasted into each service, and nobody can say what a team spent or which model answered.
  • Switching a model means a code change in every client.

What Venom adds

It doesn't replace vLLM, Ollama or your providers. It sits in front of them.

  • The orchestrator
    One OpenAI-compatible endpoint, streamed without buffering. Route aliases map to whatever serves them today, so a client never changes when a model does. Docs
  • The router
    Keeps a session on the backend that holds its prompt cache, skips targets that are down or out of quota, and tries several keys per provider by priority. Docs
  • The daemon
    The orchestrator and its dashboard on your own machine, in front of Ollama, vLLM or llama.cpp, dialling out to Venom Cloud. Docs
  • Keys and budgets
    A key per person, service or CI job, with its own rate limits and spend budget, checked before the request goes anywhere. Docs
  • Logs and metrics
    Every request with its key, model, target, tokens, cost and latency, a one-line query language over them, and Prometheus metrics with a Grafana dashboard. Docs
  • A failover drill
    Pick a provider, key or daemon to take down and see, without taking anything down, what fails over, to what, and at what price. Docs

One endpoint in front of everything

Clients call one URL. The endpoint picks a target in the order your route sets, keeps sessions where their cache is warm, and moves on before the first byte when a target fails.

Your clients

Any OpenAI SDK, the venom CLI, editors and agents.
  • /v1/chat/completions
  • /v1/responses
  • /v1/embeddings

One endpoint

Venom Cloud, or the orchestrator on your own server. It keeps a session on the backend that holds its prompt cache, checks each key's budget, logs every request and moves to the next target before the first byte when one fails.

Your GPUs

Daemons that dial out from your machines.
  • vLLM
  • SGLang
  • llama.cpp
  • Ollama

Providers on your keys

Several keys per provider, tried by priority.
  • OpenAI
  • Z.ai
  • Qwen
  • TypeSafe

Public nodes you allowlist

Optional. Checked by audits and measured on real jobs.

Works with the code you have

Speaks the OpenAI API

  • /v1/chat/completions
  • /v1/responses
  • /v1/embeddings
  • /v1/completions
  • /v1/models

Providers on your keys

  • OpenAI
  • Z.ai
  • Qwen (DashScope)
  • TypeSafe

Backends on your daemons

  • vLLM
  • SGLang
  • llama.cpp
  • Ollama

Change the base URL to https://cloud.usevenom.dev/v1, use a vck_ key, and keep your SDK.

Each request goes where its prompt cache is warm

The same router runs in the daemon, in front of your backends, and in Venom Cloud, across your daemons. It keeps a session on the GPU that holds its prompt and fails over before the first byte.

  • +49 µs8

    added per request at p50, auth and routing included

  • 100%11

    of a session's requests on its warm daemon at 10% load, up from 61-64%

  • 7 µs12

    for the cloud to pick among 10 daemons, p50

  • 1,00013

    concurrent streams with 0 errors and nothing buffered

Run any model, without an account or telemetry

venom runs the model you choose, local or hosted, with no account, and it ships with telemetry and update checks off. Here is how that looks next to Claude Code and Qwen Code.

  • Models1,2,3,4

    venom CLI
    Any OpenAI-compatible endpoint: Ollama, vLLM, SGLang, llama.cpp, Venom Cloud or a provider
    Claude Code
    ~Claude models, through Anthropic or Amazon Bedrock, Google Cloud and Microsoft Foundry
    Qwen Code
    OpenAI, Anthropic, Gemini and Qwen APIs, any OpenAI-compatible endpoint, local models
  • Account needed5,2,4

    venom CLI
    None. Point it at a model and start
    Claude Code
    A Claude Pro, Max, Team, Enterprise or Console account, or a cloud provider
    Qwen Code
    A provider key, or a local model
  • With the network unplugged1,2

    venom CLI
    Works, with a local model
    Claude Code
    –Docs list an internet connection as required
    Qwen Code
    Not checked
  • Usage telemetry by default1,6,7

    venom CLI
    Off. No telemetry and no update checks
    Claude Code
    On with the Claude API (no code or prompts), off on cloud providers; one variable turns it off
    Qwen Code
    On; one setting turns it off
+49 µs8

added per request, p50

Auth, alias resolution, quirks, usage accounting and session affinity on the path. Streaming adds +65 µs p50, and nothing is buffered.

make bench on an Apple M2 Pro with a local fake upstream.

0% → 95%9

cached on a new session's first request

Sessions share a small pool of OpenAI cache keys, so a new session finds the common prompt prefix already cached.

Live, eight new sessions with the same 5k-token prefix.

97.6%10

pass@1 on Venom's eval suite

42 tasks, 3 trials each: 95.2% passed all 3. The safety slice passed 18 of 18 with no permission-engine violations. 126 trials cost $1.23, with 74.4% of input tokens cached.

Self-authored tasks, gpt-5.4-mini through the orchestrator. Not a public benchmark.

The full comparison, with every speedup

Sources

  1. 1.Venom README: the CLI turns off telemetry, update checks and downloads, and the orchestrator calls only its configured upstreams. README.md:32-33.
  2. 2.Claude Code setup docs: requires a Pro, Max, Team, Enterprise or Console account, or a third-party provider (Amazon Bedrock, Google Cloud, Microsoft Foundry); "internet connection required"; native installs update in the background, package-manager installs don't by default. code.claude.com/docs/en/setup. Checked 2026-10-18.
  3. 3.Claude Code LLM gateway docs: Anthropic "doesn't support routing Claude Code to non-Claude models through any gateway." code.claude.com/docs/en/llm-gateway. Checked 2026-10-18.
  4. 4.Qwen Code README: originally based on Google Gemini CLI v0.8.2, developed independently since v0.1; Apache-2.0; supports OpenAI, Anthropic, Gemini and Qwen APIs and local models (Ollama, vLLM). github.com/QwenLM/qwen-code. Checked 2026-10-18.
  5. 5.The venom CLI, the orchestrator and the cloud are MIT licensed; the CLI needs no account. cli/package.json:7, docs/release.md:144.
  6. 6.Claude Code data-usage docs: metrics are on by default with the Claude API (they never include code, prompts or file paths), off by default on Bedrock, Google Cloud and Foundry; DISABLE_TELEMETRY or CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC turns them off. code.claude.com/docs/en/data-usage. Checked 2026-10-18.
  7. 7.Qwen Code settings docs: privacy.usageStatisticsEnabled defaults to true; general.enableAutoUpdate (update checks and installs on startup) defaults to true; approval modes plan, default, auto-edit, auto and yolo. qwenlm.github.io/qwen-code-docs/en/users/configuration/settings. Checked 2026-10-18.
  8. 8.make bench, Apple M2 Pro, a local fake upstream, a ~5 KB chat request with a tool; the added time includes the extra loopback hop. Re-runs on a loaded machine measured +59 to +63 µs. services/go-orchestrator/AGENTS.md "chat body with a tool; non-stream: one JSON response;".
  9. 9.OpenAI prompt_cache_key pooled across sessions, measured live: eight new sessions with the same 5k-token prefix. CHANGELOG.md "live, eight new sessions with the same".
  10. 10.Venom eval baseline, 2026-10-09: 42 self-authored tasks in 8 categories, k=3 (126 trials), venom-default routed to gpt-5.4-mini through a local orchestrator. Not a public benchmark and not a head-to-head with other agents. evals/results/2026-10-09T08-33-43Z-23be.json, evals/AGENTS.md "`apply_patch` and no `edit`. Results: pass@1".
  11. 11.Bounded-load session owner, simulated (npm run sim:bounded): 10 daemons × 4 slots, Poisson arrivals, 1,000 Zipf-popular sessions (s = 0.8 and 1.1), 50k requests. A simulation, not a production measurement. docs/cloud-mesh.md "(`bun run sim:bounded`, `sim/bounded-load.ts`). The setup".
  12. 12.Venom Cloud's route() wall-clock time on Node 24 for 10 eligible daemons (65 µs p50 for 100). docs/cloud-mesh.md "Only the `route()` timings are wall-clock measurements", docs/cloud-mesh.md "`route()` itself takes".
  13. 13.Load test against a fake vLLM upstream (300 ms TTFT, 100 tokens/s) on a shared M2 Pro; numbers are at 1,000 concurrent streams. services/go-orchestrator/AGENTS.md "tokens/s per stream,", services/go-orchestrator/AGENTS.md "**Streaming levels**".

Ask your request log a question in one line

Filter by model, key, status, latency, cost or daemon, then pipe into counts, percentiles, top lists or a time series. The console colours and completes the query as you type.

model:gpt-* status:5xx latency:>2s | p95(latency) by provider
Example result: p95 latency of slow 5xx requests per provider
providerp95(latency)requests
openai3.42 s41
qwen2.87 s18
zai2.31 s7

Example output. Runs over your org's request log in the console.

More you can ask

  • key:ci-* cost:>$0.05 @today

    expensive calls from CI keys today

  • daemon:gpu-box ttft:<300ms

    fast first tokens on one daemon

  • | top 5 model by sum(cost)

    the five costliest models

  • | timeseries 5m count by status

    requests per status, as a chart

A coding agent on your own models

The venom CLI works in your terminal on your real repository and speaks the same API. Point it at your mesh, and every call shows up in the log under your key's budget.

  • Four permission modes, with shell commands parsed and judged part by part
  • Subagents, plan mode and /undo checkpoints
  • Claude Code projects, plugins, hooks and MCP servers load as they are
  • With a local model it works offline
Download venomChoose your platform belowAbout the venom CLI
~/work/api · venom
Illustration of a session. Your model, your rules.

Your keys pass through at cost

Traffic on your own provider keys and your own daemons carries no markup. When Venom supplies the inference, on its own provider keys or the managed public mesh, you pay the provider's cost plus 20%.

  • The orchestrator, the daemon and the venom CLI are free downloads
  • Venom Cloud plans are a flat fee for the organisation, with unlimited collaborators
  • Budgets and spend caps per key, key group and organisation

Questions people ask first

What is a daemon?
The Venom orchestrator, a Go binary, running in mesh mode on a machine you own. It sits in front of vLLM, SGLang, llama.cpp or Ollama, and any provider keys you give it, connects out to the cloud, and serves the requests routed to it.
Does it work with the code I already have?
If it speaks the OpenAI API, yes. Set the base URL to https://cloud.usevenom.dev/v1, use a vck_ key from the console, and keep your SDK.
What happens when a provider goes down?
The router moves to the next target in the route before the first byte reaches the client, and the response says why in the x-venom-fallback header. A target that keeps failing has its circuit breaker opened and is skipped until it recovers.
Can I run it without Venom Cloud?
Yes. The orchestrator runs on your own server with Docker Compose or Kubernetes, and the venom CLI can talk to it directly. See Deploy.
Can I mix local and cloud models in one route?
Yes. A route lists its targets in order: your daemons, your provider keys, and public nodes if you allow them. The cost routing guide shows an alias that tries the cheapest first.

Send your first request to your own hardware today

Sign up, connect a daemon or add a provider key, and point any OpenAI client at one URL.