Voice agents: calls in venom serve
Configure voice agents in venom.yaml, take calls on the browser test line, and read each call's events, transcript and recording over the calls API.
venom serve answers phone-style calls with a voice agent: a harness agent with its own business tools, speaking
through the orchestrator's streaming speech-to-text (/v1/realtime?intent=transcription) and text-to-speech
(/v1/audio/speech). Each call gets its own harness session. Only the agent's tools are active, and the permission
gate is on. The design is in docs/voice-agents.md.
Configure
Voice agents live under voice in venom.yaml. Dictation's voice.model and voice.language are unrelated and keep
working. Nothing starts unless voice.agents names at least one agent.
voice:
max_calls: 8 # calls at once; past it a new call is refused (409 max_calls)
retention_days: 90 # days of calls kept in ~/.venom/calls (0 keeps everything)
agents:
receptionist:
dir: ./agents/receptionist
routes: # which agent answers which number; first match wins
- { to: "6001", agent: receptionist }
- { to: "*", agent: receptionist }
browser:
origins: ["http://127.0.0.1:3000"]| Key | What it does |
|---|---|
voice.max_calls |
Calls at once (default 8). |
voice.retention_days |
Days of stored calls kept. Older days are pruned at start-up and daily. |
voice.agents.<name>.dir |
The agent's directory. A relative path is resolved against where venom serve starts. |
voice.routes |
{to, agent} pairs: an E.164 number, an Asterisk extension, or *. |
voice.browser.origins |
Origins allowed to open the browser line's media socket. The default is VENOM_CONSOLE_ORIGIN, else http://127.0.0.1:3000 and http://localhost:3000. |
voice.asterisk |
Self-hosted SIP: ari_url, ari_user, app, rtp_bind, rtp_ports, codec (pcmu or pcma), trunk. |
voice.twilio |
Twilio Media Streams: public_url (a public HTTPS URL that reaches venom serve) and from. |
From a project's venom.yaml, voice.agents, voice.routes, voice.asterisk, voice.twilio and voice.browser only
apply once the project is trusted (venom trust), because an agent's tools module runs and telephony settings decide
where calls go.
An agent directory holds three files:
agent.yaml:model(a chat alias),language,stt: {model}(a realtime STT alias),tts: {model, voice}, and optionallygreeting,vad,turn,interruption,fillers,compliance(recording: truerecords both sides) andlimits(maxCallMs,maxUsd,silenceHangupMs).instructions.md: the system prompt. It is fixed per agent, so every call shares the prompt cache.tools.ts(optional): a pi extension module whose default export registers the agent's business tools.
The browser test line
POST /v1/calls with {"agent": "receptionist", "transport": "browser"} answers with the call and a media object:
a WebSocket URL for GET /v1/calls/{callId}/media and a ticket. The ticket is single use, expires after 60 s, and only
opens that one call. The page opens the socket with ?ticket= from an allowed origin. It sends 16 kHz PCM16 microphone
audio and receives 24 kHz PCM16 speech, plus hello, mark, clear and end messages. It answers marks with
mark_played and can send dtmf and hangup (docs/voice-agents.md §5.4).
Venom Line's media bridge
POST /v1/calls with "transport": "bridge" is what the orchestrator's media bridge calls when a phone provider
streams a Venom Line call to it. It works like the browser line, without the Origin check: the caller's and the
line's numbers go in from and to, output_rate (8000, 16000 or 24000) sets the agent audio's rate, and line
carries the cloud's per-call settings (opening, voice, tools, max_call_s, max_usd). On the media socket
the server also sends tool and transfer messages, which the bridge answers with tool_result and
transfer_result. See Venom Line.
Twilio
Twilio is optional and off until you set it up. It needs TWILIO_ACCOUNT_SID and TWILIO_AUTH_TOKEN in venom serve's
environment (.env), plus voice.twilio.public_url: a public HTTPS URL, such as a tunnel or reverse proxy you run,
that reaches venom serve. Without all three, venom never contacts Twilio.
- Point the number's voice webhook at
{public_url}/v1/voice/twilio/incoming(HTTP POST). - Add a
voice.routesentry for the number (to: "+15550001111").
Each webhook is checked against X-Twilio-Signature in constant time; a request without a valid signature gets a 403.
The answer connects the call to a Media Stream at /v1/voice/twilio/media with a single-use ticket. Audio is µ-law
8 kHz both ways. When the caller talks over the agent, venom sends clear, and Twilio's mark echoes show how much of
the reply the caller heard. Closing the stream hangs up.
Outbound calls (POST /v1/calls with "transport": "twilio" and to) come from voice.twilio.from and use
/v1/voice/twilio/status for status callbacks. Answering-machine detection is requested for agents with the
collections-us pack. A warm transfer updates the live call with <Dial>. When there is a whisper for the person
taking the call, Twilio fetches it once from /v1/voice/twilio/whisper/{token}, which plays the clip
from GET /v1/voice/clips/{clip}. Both tokens are single use and expire after 2 minutes.
Twilio reaches venom serve through your tunnel, so requests carry the public host name. venom serve accepts that host
only on /v1/voice/*.
Asterisk (self-hosted SIP)
Asterisk handles SIP: phones, trunks, registration. venom serve drives it over ARI and exchanges G.711 audio with it over RTP. Nothing reaches a phone network unless you configure a trunk.
- Put
VENOM_ARI_PASSWORDin.env(any long random value). - Start Asterisk:
docker compose --profile sip up -d asterisk. It uses the host's network (Linux, or Docker Desktop with host networking turned on). ARI listens on127.0.0.1:8088and SIP on127.0.0.1:5060. - Add
voice.asteriskto venom.yaml and start venom serve with the same.env:
voice:
asterisk:
ari_url: http://127.0.0.1:8088
rtp_ports: 40000-40199 # venom's side; Asterisk's own range is 10000-10199
codec: pcmu # or pcma
trunk: my-trunk # for outbound calls: PJSIP/<number>@my-trunkTo test, register a softphone on the same machine as user 6001 (password in deploy/asterisk/pjsip.conf, change
it) at 127.0.0.1:5060, and dial any number. The dialplan hands every call to venom, and voice.routes picks the
agent by the dialled number. A call with no route is hung up.
To reach the phone network, fill in the commented-out my-trunk sections in pjsip.conf with your SIP provider's
details. To take calls from other machines, change the transport's bind there and open 5060/udp and Asterisk's RTP
range in your firewall yourself. Neither happens by default.
Outbound calls (POST /v1/calls with "transport": "asterisk" and to) originate through the trunk. Agents with the
collections-us pack run Asterisk's AMD() first, and the result shows up as the call's answering-machine status.
A warm transfer puts the caller on hold music, rings the target, plays the whisper to the target alone, then joins
the two and the agent leaves.
The calls API
All routes need the bearer token except the media socket, which needs its ticket. Without voice agents they answer
404 voice_not_configured.
| Route | What |
|---|---|
GET /v1/calls |
Live and stored calls, newest first; state, agent, limit, before filter them. |
POST /v1/calls |
Start a call: browser, or asterisk/twilio with to. |
GET /v1/calls/{id} |
One call: summary, compliance events, last handoff, per-turn latency metrics. |
GET /v1/calls/{id}/events |
Server-sent events, one per call event; Last-Event-ID or after resumes. An ended call replays its stored events. |
GET /v1/calls/{id}/transcript |
Caller finals and, for the agent, what the caller actually heard. |
GET /v1/calls/{id}/recording |
Stereo 8 kHz WAV (caller left, agent right) when the agent records. |
POST /v1/calls/{id}/hangup |
Hang up a live call. |
POST /v1/calls/{id}/transfer |
Hand a live call to the agent's handoff target. |
GET /v1/voice/agents |
The configured agents, their transports and compliance settings. |
The Python client has the same calls: calls, create_call, call, call_events, call_transcript,
call_recording, hangup_call, transfer_call and voice_agents.
Where calls are kept
Each call is stored in ~/.venom/calls/<day>/<call id>/: call.json, events.jsonl, transcript.jsonl and
recording.wav. Directories are mode 0700 and files 0600. Secrets in transcript and event text are redacted before
they are written, as in session logs. Nothing leaves the machine.