Add a provider key and route by cost
Add a cloud provider to your orchestrator next to local models, make an alias that tries the cheapest target first, choose it per key, request or gear, and read back what it cost.
This recipe starts from an orchestrator serving local models (offline setup or the orchestrator quick start) and adds a cloud provider as a cheaper or stronger option, without sending anything to it until you choose.
1. Add the key
Provider keys live in the orchestrator's environment only, never in the CLI's or in a config file. The shipped config already has upstreams for these, each disabled while its variable is empty:
| Variable | Upstream (name, kind) |
Notes |
|---|---|---|
OPENAI_API_KEY |
openai, openai |
OPENAI_API_KEY_2 and more as fallback keys with api_key_envs |
QWEN_API_KEY |
qwen, dashscope |
Alibaba Cloud Model Studio; QWEN_BASE_URL picks the region |
ZAI_API_KEY |
zai, zai |
Z.ai GLM; ZAI_BASE_URL picks pay-as-you-go or the Coding Plan |
MISTRAL_API_KEY |
mistral, mistral |
commented out: uncomment the upstream too |
MINIMAX_API_KEY |
minimax, minimax |
commented out: uncomment the upstream too |
ANTHROPIC_API_KEY |
anthropic, anthropic |
commented out: uncomment the upstream too |
From a source checkout, put the key in the gitignored root .env and restart the stack:
# .env
QWEN_API_KEY=sk-…make up SERVICES=orchestratorFor a release binary or the desktop daemon, set it in the orchestrator's environment and restart it. A config
change alone needs only a reload (POST /admin/v1/reload or SIGHUP), but the environment is read at start.
Check it is enabled and healthy:
curl -s http://127.0.0.1:8787/admin/v1/upstreams -H "Authorization: Bearer $VENOM_ADMIN_TOKEN" \
| jq '.[] | {name, disabled, healthy}'Another provider with an OpenAI-compatible API is an upstream of its own; the upstream kinds
table says which kind to use, and examples has Azure, LiteLLM and
others.
2. An alias that tries the cheapest first
routes:
- alias: venom-thrifty
strategy: cheapest
targets:
- {upstream: openai, model: gpt-5.4-mini}
- {upstream: qwen, model: qwen3.8-flash}
- {upstream: ollama, model: "${VENOM_OLLAMA_MODEL:-qwen3:8b}"}cheapest orders the targets by effective price, a 3:1 blend of input and output price, computed when the route
table is built: local targets first (they cost $0), then the cloud ones by price, unpriced ones last, ties in the
order written. Health checks, circuit breakers and fallback work as on any route, so a down target is skipped and
the next cheapest serves. Reload, then look at the order it chose:
curl -s -X POST http://127.0.0.1:8787/admin/v1/reload -H "Authorization: Bearer $VENOM_ADMIN_TOKEN"
curl -s http://127.0.0.1:8787/admin/v1/pricing -H "Authorization: Bearer $VENOM_ADMIN_TOKEN"GET /admin/v1/pricing lists every model's price and source, the blended price cheapest ranks by, each route's
targets in price order, and enabled cloud models nothing prices.
Other strategies that weigh cost:
| Strategy | Order | Tune with |
|---|---|---|
cheapest |
by price, fixed at load | the prices (below) |
balanced |
by a weighted sum of price and live time to first token | balance: {price: 0.3, latency: 0.7} |
learned |
the cheapest target whose reported session success is within margin of the best |
learned: {min_samples: 20, margin: 0.10}; outcomes from POST /admin/v1/sessions/{id}/outcome |
fastest |
by live time to first token only | explore |
All of them are in routes and strategies.
3. Prices you control
Prices come from the offline price table (prices.yaml next to the config, or VENOM_PRICES), never fetched at
runtime. Price a model the table does not know, or one you pay a negotiated rate for, under pricing::
pricing:
source: [custom, builtin]
models:
litellm/my-model: {input: 0.2, cached_input: 0.05, output: 0.8} # your own rate, USD per 1M tokens
azure/my-gpt: {base_model: gpt-5.4} # priced as another modelA target can also carry its own input, cached_input and output, and a route its base_model
(pricing).
4. Choose it per key, request or gear
Clients keep asking for the aliases they know; you decide which of them goes cheap.
Per request, call the alias by name: "model": "venom-thrifty".
Per key, without changing the client, re-target its usual aliases with a route profile and make it that key's default:
profiles:
thrifty:
description: local first, then the cheapest cloud model
routes:
venom-default: [ollama/qwen3:8b, qwen/qwen3.8-flash, openai/gpt-5.4-mini]
venom-fast: [ollama/qwen3:4b, qwen/qwen3.8-flash]curl -s -X PATCH http://127.0.0.1:8787/admin/v1/keys/<id> \
-H "Authorization: Bearer $VENOM_ADMIN_TOKEN" -H "Content-Type: application/json" \
-d '{"profile":"thrifty"}'A profile replaces the alias's targets in the order given and keeps its strategy. A client can also pick it per
request with x-venom-profile: thrifty (route profiles).
In the venom CLI, a gear in venom.yaml puts the alias one keystroke away:
gears:
thrifty:
key: 10
description: cheapest target first
model: venom-thrifty
thinking: lowOr send the profile from a gear with profile: thrifty (your own gears).
5. Read back what it cost
Every answer names the target that served it:
curl -s http://127.0.0.1:8787/v1/chat/completions -H "Authorization: Bearer $VENOM_API_KEY" \
-H "Content-Type: application/json" -D - -o /dev/null \
-d '{"model":"venom-thrifty","messages":[{"role":"user","content":"ping"}]}' | grep -i '^x-venom'x-venom-upstream and x-venom-model say who served; x-venom-fallback says why earlier targets did not. Over
time:
curl -s "http://127.0.0.1:8787/admin/v1/spend?from=2026-11-01&to=2026-11-30" -H "Authorization: Bearer $VENOM_ADMIN_TOKEN"returns totals and breakdowns by key, team, model and day, with the costliest sessions; the console's Usage page
shows the same. Put a budget on a key so a mistake cannot run up a bill: budget: {daily_usd: 5} on
POST /admin/v1/keys or PATCH /admin/v1/keys/{id}. Requests that reach only local models cost $0 and are never
refused (keys and budgets).
Troubleshooting
| Symptom | Cause and fix |
|---|---|
The upstream shows disabled |
Its key variable is empty in the orchestrator's environment. Set it and restart; a reload is not enough |
cheapest puts a cloud model last |
Nothing prices it. Add it under pricing.models; GET /admin/v1/pricing lists unpriced models |
403 unpriced_model |
A key with a hard budget reached a cloud target with no price. Price the model |
400 unknown_profile |
The profile name is not in profiles:; the error lists the configured ones |
Requests to venom-default now reach the provider |
The shipped routes have cloud fallbacks. Remove the cloud target, or use a profile that lists only local targets |
The load warns about ttft_timeout |
A route's first-byte deadline is not shorter than defaults.retry_budget, so no fallback could start. Lower it |