Skip to content
Venom daemon v0.2.19 · no Docker · dial-out only

Your GPUs, one API.One click to start.

The daemon puts an OpenAI-compatible endpoint in front of the models on your machine. Install it, click to start, and its dashboard opens in your browser. Connect it to Venom Cloud when you want your team to reach it.

macOS, Linux and Windows (beta). Every download

How it works

Three steps, no terminal needed

  1. 1

    Install

    Download the app or run the one-line installer. It checks the SHA-256 and needs no admin rights.

  2. 2

    It opens your dashboard

    The daemon starts on 127.0.0.1 and opens its dashboard in your browser, already signed in. No tokens to copy.

  3. 3

    Paste a join token

    In Venom Cloud, Daemons, Connect a daemon gives you a join token. Paste it into Connect to Venom Cloud and the daemon joins your org. Skip this step to use it on its own.

Install

One line, or one download

Each download holds the launcher, the orchestrator, the dashboard and the Node runtime the dashboard runs on. The scripts fetch only from the public releases repository, check the SHA-256, add a launch entry, then start the daemon and open its dashboard.

macOS and Linux

curl -fsSL https://raw.githubusercontent.com/elishahbiz/venom-releases/main/install-daemon.sh | sh

Installs to ~/.venom and adds Venom Daemon to Applications (macOS) or your app launcher (Linux).

Windows, in PowerShell Beta

irm https://raw.githubusercontent.com/elishahbiz/venom-releases/main/install-daemon.ps1 | iex

Adds a Venom Daemon shortcut to the Start Menu. On Windows, stopping the daemon ends it at once.

The macOS app

Unzip it and double-click Venom Daemon. The app is not notarized by Apple, so the first time, right-click it and choose Open, or allow it in System Settings, Privacy & Security, Open Anyway. The one-line installer doesn't need this step.

Every download

An archive unpacks to venom-daemon/; run bin/venom-daemon (on Windows, bin\venom-daemon.exe.

  • macOS (Apple silicon) app
    Venom-Daemon-0.2.0-macos-arm64.zip
    Download
  • macOS (Apple silicon) archive
    venom-daemon-0.2.0-darwin-arm64.tar.gz
    Download
  • macOS (Intel) app
    Venom-Daemon-0.2.0-macos-x64.zip
    Download
  • macOS (Intel) archive
    venom-daemon-0.2.0-darwin-x64.tar.gz
    Download
  • Linux x64 archive
    venom-daemon-0.2.0-linux-x64.tar.gz
    Download
  • Linux arm64 archive
    venom-daemon-0.2.0-linux-arm64.tar.gz
    Download
  • Windows x64 archiveBeta
    venom-daemon-0.2.0-windows-x64.zip
    Download

Checksums: SHA256SUMS. Release notes and every asset: the releases page.

Day to day

venom-daemon          # start, or open the dashboard
venom-daemon status
venom-daemon logs
venom-daemon stop

Remove it

venom-daemon uninstall          # keeps your data
venom-daemon uninstall --purge  # leaves Venom Cloud, deletes the data

Who it's for

Two ways people run it

Privacy-first orgs on their own GPUs

Run the models next to your data. Your team reaches them through Venom Cloud with keys, budgets and logs, and routes marked private only ever reach your own daemons.

GPU owners joining the mesh

The same daemon can serve the public mesh. The cloud checks the work with measured latency, audits and canaries. Payouts to hosts come in a later phase; for now, public hosting starts with a waitlist.

What it runs

Everything on your machine, nothing in a container

The Venom orchestrator

An OpenAI-compatible gateway on 127.0.0.1: model aliases, fallback, circuit breaking and session-sticky cache affinity, streaming with no buffering, usage and keys in SQLite.

Your backends

Ollama, vLLM, llama.cpp and SGLang. If Ollama isn't installed, the dashboard links to its official download; then it pulls the models that fit your hardware.

A local dashboard

Local models, routes, requests, spend and the Venom Cloud connection, on 127.0.0.1. Start at login is off until you turn it on.

Want a coding agent on top? The venom CLI talks to the daemon's /v1.

The router inside

What the daemon does with each request

The daemon is the Venom router on your machine. Before a request reaches Ollama, vLLM or llama.cpp, it picks the backend, holds its place in the queue and relays the stream as it is written.

  1. Radix tree of served prompts

    A new session goes to the backend that already holds its prefix5

    On a route with several self-hosted backends, the orchestrator keeps a bounded radix tree of the prompts each one served, in 512-byte blocks. A new session lands on the backend with the longest match, unless that backend is busier than the rest.

    75.5% → 79.7%

    prefix-cache hit rate, for +0.62 µs and one allocation per request

    Simulated on 3 fake vLLM targets with tight caches.

  2. Retry budgets and backpressure

    Retries that can't turn an outage into a storm12,13

    Same-upstream retries draw on a budget of 20% of first attempts, and a retry the OpenAI SDK is already making isn't retried again underneath it. A connection reset counts as backpressure, not a dead upstream, so a cold start no longer trips the breaker.

    30 → 13

    upstream calls for 10 requests to a failing upstream; breaker trips on a cold start went from 4 of 14 to 0

    Orchestrator tests and load tests on a shared M2 Pro.

  3. Streaming without buffering

    The first token reaches you when the model writes it14,15

    Bytes go to the client as the upstream writes them, and every stream tells proxies such as nginx not to hold events back. Request bodies are rewritten by scanning and splicing the JSON in place, not by decoding and re-encoding it.

    5.05 s → 0.01 s

    to the first event behind nginx with gzip; a 525 KB body rewritten in 285 µs instead of 1.60 ms

    make smoke against the prod image, and make bench on an M2 Pro.

  4. Weighted priority classes

    Interactive requests get most of a busy backend's slots16

    When a backend is at its concurrency limit, waiting requests are served by class: interactive, then merge checks, then background work, by smooth weighted round robin at 8:3:1, first come first served within a class.

Every routing technique, in the daemon and the cloud

Compared

More than a model server on its own

Keep running Ollama or vLLM. The daemon sits in front of them and your provider keys, and adds routing, fallback, cache affinity and per-key limits for 49 µs a request at p50.

  • OpenAI-compatible API1,2,3

    Venom daemon
    Chat, responses, completions, embeddings and models, with streaming
    Ollama on its own
    ~A subset of the OpenAI API
    vLLM on its own
    Completions, chat and more
  • Several backends behind one endpoint1,2,4

    Venom daemon
    vLLM, SGLang, llama.cpp, Ollama and cloud providers, by model alias
    Ollama on its own
    Its own models, plus Ollama's hosted cloud models
    vLLM on its own
    One server; the vLLM project ships separate routers for many
  • Fallback and circuit breaking1,2,4

    Venom daemon
    Fallback before the first byte, circuit breakers per target
    Ollama on its own
    Not documented
    vLLM on its own
    ~In the separate vllm-project/router
  • Prefix-cache affinity1,5,2,3,4

    Venom daemon
    Session-sticky routing, and longest-prefix routing for new sessions
    Ollama on its own
    Not documented
    vLLM on its own
    ~Automatic prefix caching on an instance; session and cache-aware routing in the separate routers
  • Client keys, limits and budgets1,2

    Venom daemon
    Per-key rate and token limits and spend budgets, usage and cost in SQLite
    Ollama on its own
    The local API needs no authentication
    vLLM on its own
    Not checked
  • Listens on localhost by default1,2

    Venom daemon
    127.0.0.1
    Ollama on its own
    127.0.0.1:11434
    vLLM on its own
    Not checked
  • Reachable by your team without an inbound port6,2,3

    Venom daemon
    Optional: it dials out to Venom Cloud after you paste a join token
    Ollama on its own
    Not documented
    vLLM on its own
    Not documented
+49 µs7

added per request, p50

Auth, alias resolution, quirks, usage accounting and session affinity on the path. Streaming adds +65 µs p50, and nothing is buffered.

make bench on an Apple M2 Pro with a local fake upstream.

1,0008

concurrent streams, 0 errors

+0.4 ms TTFT at p50, throughput 0.997× of the upstream alone, 163 MB peak memory.

Fake vLLM upstream on a shared M2 Pro.

0% → 95%9

cached on a new session's first request

Sessions share a small pool of OpenAI cache keys, so a new session finds the common prompt prefix already cached.

Live, eight new sessions with the same 5k-token prefix.

0% → 88-95%10

prompt reused on llama.cpp

The orchestrator leaves slot choice to builds with the host-RAM prompt cache instead of pinning, so interleaved sessions keep their cache.

llama.cpp b11459, measured through the orchestrator.

75.5% → 79.7%5

prefix-cache hit rate, simulated

A new session goes to the backend that holds the longest prefix of its prompt, unless that backend is busy.

Simulation on 3 fake vLLM targets with tight caches.

7 µs11

for Venom Cloud to pick a daemon, p50

11 µs at p95 across 10 eligible daemons, 65 µs across 100. Routing is not where a request's time goes.

Wall clock on Node 24.

The full comparison, with every speedup

Sources

  1. 1.The orchestrator: model aliases over vLLM, SGLang, llama.cpp, Ollama and cloud providers, fallback before the first byte, circuit breakers, session-sticky routing for KV-cache reuse, per-key limits and budgets, usage and keys in SQLite. README.md:87.
  2. 2.Ollama docs: "supports a subset of the OpenAI API"; binds 127.0.0.1:11434 by default; the local API "does not require authentication"; hosted cloud models are also offered. docs.ollama.com (openai-compatibility, faq, api/authentication). Checked 2026-10-18.
  3. 3.vLLM docs: an OpenAI-compatible server (Completions, Chat and more) and Automatic Prefix Caching, which reuses the KV cache of queries that share a prefix. docs.vllm.ai (openai_compatible_server, automatic_prefix_caching). Checked 2026-10-18.
  4. 4.The vLLM project's separate routers: production-stack (session-ID routing) and vllm-project/router (cache-aware load balancing, circuit breakers, retries). github.com/vllm-project/production-stack, github.com/vllm-project/router. Checked 2026-10-18.
  5. 5.Longest-prefix routing for new sessions, simulated on 3 fake vLLM targets with tight caches and 8 prompt families. A simulation, not a production measurement. CHANGELOG.md "unless that target is busy against the others (SGLang's balance".
  6. 6.Venom Cloud's routing: direct (a 307 redirect with a single-use ticket) or relayed through the cloud; daemons only dial out. docs/cloud-mesh.md:36-44.
  7. 7.make bench, Apple M2 Pro, a local fake upstream, a ~5 KB chat request with a tool; the added time includes the extra loopback hop. Re-runs on a loaded machine measured +59 to +63 µs. services/go-orchestrator/AGENTS.md "chat body with a tool; non-stream: one JSON response;".
  8. 8.Load test against a fake vLLM upstream (300 ms TTFT, 100 tokens/s) on a shared M2 Pro; numbers are at 1,000 concurrent streams. services/go-orchestrator/AGENTS.md "tokens/s per stream,", services/go-orchestrator/AGENTS.md "**Streaming levels**".
  9. 9.OpenAI prompt_cache_key pooled across sessions, measured live: eight new sessions with the same 5k-token prefix. CHANGELOG.md "live, eight new sessions with the same".
  10. 10.llama.cpp b11459 with the host-RAM prompt cache, interleaved sessions, measured through the orchestrator. CHANGELOG.md "pinning each session to a slot with `id_slot`".
  11. 11.Venom Cloud's route() wall-clock time on Node 24 for 10 eligible daemons (65 µs p50 for 100). docs/cloud-mesh.md "Only the `route()` timings are wall-clock measurements", docs/cloud-mesh.md "`route()` itself takes".
  12. 12.Retry budget test: 10 requests to an upstream that always answers with a 503 error, max_attempts 3 (server.TestRetryBudgetCapsRetries). services/go-orchestrator/AGENTS.md "`server.TestRetryBudgetCapsRetries` (".
  13. 13.Load test: a fresh orchestrator hit by 1,000 streams at once, 14 interleaved trials, before and after connection resets were treated as backpressure. services/go-orchestrator/AGENTS.md "**Failure behaviour under load**".
  14. 14.make smoke STAGES=prod: nginx 1.30.5 with gzip in front of the prod image, a ~5 s stream; time to the first event with and without the orchestrator's X-Accel-Buffering header. services/go-orchestrator/AGENTS.md "`make smoke STAGES=prod` checks it end to end".
  15. 15.Request rewrite on a ~525 KB body (a long agent session): a JSON scan and splice against a decode and re-encode, make bench, Apple M2 Pro. services/go-orchestrator/AGENTS.md "Request rewriting on a".
  16. 16.The orchestrator's priority classes: interactive, blocking-merge and background requests share an upstream's slots by smooth weighted round robin, 8:3:1. services/go-orchestrator/AGENTS.md:1668-1671.

Hosting on the public mesh pays nothing yet: payouts to hosts are a later phase.

Security

Dial-out only. Keys stay local.

  • Listens on 127.0.0.1 only; nothing on your network can reach it
  • Connects out to Venom Cloud only after you paste a join token, and never opens an inbound port
  • Provider keys and model credentials stay on the machine
  • Its own tokens are generated on first run and stored readable by you only
  • Disconnect in the dashboard drains running jobs and deletes the cloud credential
Checked downloads

Every release publishes SHA256SUMS, and the installers refuse a file that doesn't match.

No account needed

The daemon works on its own. Venom Cloud is optional.

Works offline

With a local model, nothing leaves your machine.

FAQ

Questions people ask first

Do I need Docker?
No. The daemon runs as two processes on your machine: the orchestrator and its dashboard. Docker stays an option for servers.
Do I need Venom Cloud?
No. The daemon works on its own as a local OpenAI-compatible endpoint. Connect it to Venom Cloud when your team should reach it, and disconnect any time.
Where do models come from?
From the backend you run. With Ollama, the dashboard pulls the recommended models for your hardware. If Ollama isn't installed, the dashboard links to its official download; nothing is installed without you.
Can I earn money by hosting?
Not yet. Payouts to public hosts are a later phase. Join the host waitlist and we will tell you when hosting opens.
Why does macOS warn me about the app?
The app is signed on the build machine but not notarized by Apple. Right-click it and choose Open the first time, or use the one-line installer, which needs no extra step.
What does Windows beta mean?
It works, with two limits: stopping the daemon ends it at once, so running jobs are cut, and the launcher may show a console window briefly.

Serve your GPUs in one click

Install the daemon, open its dashboard, and connect it to your team when you are ready.

No Docker. No inbound ports. Your keys stay on your machine.