Skip to content
Docs menu

Local or cloud models

Choose where venom's model runs (Ollama on this machine, the desktop daemon, your own orchestrator, Venom Cloud or any OpenAI-compatible provider) and point the CLI at it.

View .md

venom talks to one OpenAI-compatible endpoint. Three settings decide which one and what it asks for:

Variable What it sets Default
VENOM_BASE_URL The endpoint, including /v1 http://localhost:8787/v1 (an orchestrator on this machine)
VENOM_API_KEY The key for that endpoint, sent as Authorization: Bearer none (no header)
VENOM_MODEL The main model venom-default

The rest of this page is about picking the endpoint. Your code, prompts and files go only to the endpoint you choose. venom makes no other calls: no telemetry, no update checks.

Pick one

Option What runs where Good for Cost
Ollama, direct Ollama on this machine, nothing else Trying venom offline in two minutes Free; needs RAM or a GPU
Desktop daemon venom's orchestrator and dashboard on 127.0.0.1, in front of Ollama Local models with routing, fallbacks, a request log and a UI, without Docker Free
Your own orchestrator The Go orchestrator in Docker or on a server, in front of vLLM, SGLang, llama.cpp, Ollama and your provider keys Teams and GPU servers; mixing local and cloud models with fallback Your hardware and your provider bills
Venom Cloud One hosted endpoint; requests go to your own daemons, trusted public nodes or provider keys No setup; keys, budgets and logs per key group Per token, by model
Any provider A provider's or gateway's OpenAI-compatible API You already have a key That provider's prices

You can switch at any time: the endpoint is read when venom starts, and sessions are stored on your machine whichever endpoint served them.

Ollama, direct

ollama pull qwen3:8b
ollama pull qwen3:4b
export VENOM_BASE_URL=http://localhost:11434/v1
venom

When the server lists no venom-* aliases, venom makes its own from the models it lists: one model serves all three tiers; with several, it ranks them by the parameter count in the id (4b, 8b, 32b) and then by size words (nano, mini, large, pro), so venom-default and venom-smart take the larger model and venom-fast (subagents, summaries) the smaller. With more than 12 models, or ids it cannot rank, it asks you to set the map yourself:

export VENOM_MODEL=qwen3:8b
export VENOM_ALIAS_MAP=venom-fast=qwen3:4b,venom-smart=qwen3:8b

Use models that emit structured tool calls (the qwen3 family does); venom doctor --probe checks it. Ollama truncates prompts longer than its context length without saying so; start it with a window that fits a coding session, for example OLLAMA_CONTEXT_LENGTH=32768 ollama serve, and set VENOM_CONTEXT_WINDOW=32768 so venom compacts in time.

The desktop daemon

The daemon is venom's orchestrator and console packaged for a desktop or a GPU box, with no Docker:

curl -fsSL https://raw.githubusercontent.com/elishahbiz/venom-releases/main/install-daemon.sh | sh

On Windows: irm https://raw.githubusercontent.com/elishahbiz/venom-releases/main/install-daemon.ps1 | iex.

It installs venom-daemon, starts the orchestrator and the console on 127.0.0.1, and opens the dashboard signed in. From the dashboard you install Ollama (a link to its download page), pull the models that fit this machine, and optionally connect the machine to Venom Cloud with a join token. To point the CLI at it:

venom-daemon status          # prints the /v1 URL and the secrets file
export VENOM_BASE_URL=http://127.0.0.1:8787/v1
export VENOM_API_KEY=…       # client_key in ~/.venom/daemon/secrets.json

The port is 8787 unless it was taken, in which case venom-daemon status shows the one it picked. The secrets file is %LOCALAPPDATA%\Venom\secrets.json on Windows.

Your own orchestrator

The orchestrator is the OpenAI-compatible gateway venom is designed around. It serves the aliases venom-default, venom-fast and venom-smart, routes each to the first healthy target (a local engine first, a cloud model as fallback), keeps a session on the same backend so its prompt cache is reused, and logs usage and cost in SQLite. Its default address is the CLI's default, so on the same machine only the key is needed:

export VENOM_API_KEY=…       # a client key the orchestrator accepts (VENOM_API_KEY in its .env)
venom doctor

From a source checkout, make up runs it in Docker and make model picks a local model; a release install runs it as venom-orchestrator. Provider keys such as OPENAI_API_KEY stay in the orchestrator's environment, never in the CLI's. The Orchestrator section covers upstreams, routes and profiles.

Venom Cloud

export VENOM_BASE_URL=https://cloud.usevenom.dev/v1
export VENOM_API_KEY=vck_…
venom doctor

The Venom Cloud quickstart has the sign-up and key steps. Separately, venom cloud login signs this machine in for the team features (the fleet view, shared sessions, the org policy, org MCP servers): it opens a device-code page in the browser and saves a key in ~/.venom/cloud.json. It does not change VENOM_BASE_URL; set that as above to send model requests to the cloud. venom cloud status shows the login, and venom cloud logout removes it.

Any OpenAI-compatible provider

export VENOM_BASE_URL=https://api.openai.com/v1
export VENOM_API_KEY=sk-…
venom

On api.openai.com venom maps venom-default, venom-fast and venom-smart to gpt-5.4-mini, gpt-5.4-nano and gpt-5.5. Elsewhere (vLLM, SGLang, llama.cpp's server, a gateway) it ranks the listed models as described for Ollama; set VENOM_MODEL and VENOM_ALIAS_MAP when it can't. Without the orchestrator you lose its fallback, routing and request log, and the key sits in the CLI's environment.

Keep the setting

Exports last one shell. Either add them to ~/.zshrc or ~/.bashrc, or save the endpoint in venom's config file, which every venom command reads:

venom config set orchestrator.base_url https://cloud.usevenom.dev/v1
venom config set model venom-default

That writes ~/.venom/config.yaml. Keys are never read from the config file: VENOM_API_KEY stays an environment variable. An environment variable wins over the config file, and --model wins over both.

Which model a session uses

Every command, the TUI and venom -p included, picks its model in this order: --model, then VENOM_MODEL (or model: in the config file), then the model /model saved last time (defaultModel in ~/.venom/agent/settings.json), then venom-default. A session you reopen with venom -c, -r or --session keeps the model it was using. Subagents and summaries use the fast tier: venom-fast when the endpoint lists it, otherwise the session's model, unless VENOM_FAST_MODEL names another.

Offline

venom --offline (or VENOM_OFFLINE=1) removes the web tools (web_fetch, web_search) and disables remote MCP servers for the run. With a local endpoint, nothing leaves the machine. With an orchestrator on this machine, venom also notices when a cloud upstream stops answering mid-session and moves the session to the local gear; see Your first gear.