Skip to content
Docs menu

Voice agents: calls in venom serve

Configure voice agents in venom.yaml, take calls on the browser test line, and read each call's events, transcript and recording over the calls API.

View .md

venom serve answers phone-style calls with a voice agent: a harness agent with its own business tools, speaking through the orchestrator's streaming speech-to-text (/v1/realtime?intent=transcription) and text-to-speech (/v1/audio/speech). Each call gets its own harness session. Only the agent's tools are active, and the permission gate is on. The design is in docs/voice-agents.md.

Configure

Voice agents live under voice in venom.yaml. Dictation's voice.model and voice.language are unrelated and keep working. Nothing starts unless voice.agents names at least one agent.

voice:
  max_calls: 8              # calls at once; past it a new call is refused (409 max_calls)
  retention_days: 90        # days of calls kept in ~/.venom/calls (0 keeps everything)
  agents:
    receptionist:
      dir: ./agents/receptionist
  routes:                   # which agent answers which number; first match wins
    - { to: "6001", agent: receptionist }
    - { to: "*", agent: receptionist }
  browser:
    origins: ["http://127.0.0.1:3000"]
Key What it does
voice.max_calls Calls at once (default 8).
voice.retention_days Days of stored calls kept. Older days are pruned at start-up and daily.
voice.agents.<name>.dir The agent's directory. A relative path is resolved against where venom serve starts.
voice.routes {to, agent} pairs: an E.164 number, an Asterisk extension, or *.
voice.browser.origins Origins allowed to open the browser line's media socket. The default is VENOM_CONSOLE_ORIGIN, else http://127.0.0.1:3000 and http://localhost:3000.
voice.asterisk Self-hosted SIP: ari_url, ari_user, app, rtp_bind, rtp_ports, codec (pcmu or pcma), trunk.
voice.twilio Twilio Media Streams: public_url (a public HTTPS URL that reaches venom serve) and from.

From a project's venom.yaml, voice.agents, voice.routes, voice.asterisk, voice.twilio and voice.browser only apply once the project is trusted (venom trust), because an agent's tools module runs and telephony settings decide where calls go.

An agent directory holds three files:

  • agent.yaml: model (a chat alias), language, stt: {model} (a realtime STT alias), tts: {model, voice}, and optionally greeting, vad, turn, interruption, fillers, compliance (recording: true records both sides) and limits (maxCallMs, maxUsd, silenceHangupMs).
  • instructions.md: the system prompt. It is fixed per agent, so every call shares the prompt cache.
  • tools.ts (optional): a pi extension module whose default export registers the agent's business tools.

The browser test line

POST /v1/calls with {"agent": "receptionist", "transport": "browser"} answers with the call and a media object: a WebSocket URL for GET /v1/calls/{callId}/media and a ticket. The ticket is single use, expires after 60 s, and only opens that one call. The page opens the socket with ?ticket= from an allowed origin. It sends 16 kHz PCM16 microphone audio and receives 24 kHz PCM16 speech, plus hello, mark, clear and end messages. It answers marks with mark_played and can send dtmf and hangup (docs/voice-agents.md §5.4).

Venom Line's media bridge

POST /v1/calls with "transport": "bridge" is what the orchestrator's media bridge calls when a phone provider streams a Venom Line call to it. It works like the browser line, without the Origin check: the caller's and the line's numbers go in from and to, output_rate (8000, 16000 or 24000) sets the agent audio's rate, and line carries the cloud's per-call settings (opening, voice, tools, max_call_s, max_usd). On the media socket the server also sends tool and transfer messages, which the bridge answers with tool_result and transfer_result. See Venom Line.

Twilio

Twilio is optional and off until you set it up. It needs TWILIO_ACCOUNT_SID and TWILIO_AUTH_TOKEN in venom serve's environment (.env), plus voice.twilio.public_url: a public HTTPS URL, such as a tunnel or reverse proxy you run, that reaches venom serve. Without all three, venom never contacts Twilio.

  1. Point the number's voice webhook at {public_url}/v1/voice/twilio/incoming (HTTP POST).
  2. Add a voice.routes entry for the number (to: "+15550001111").

Each webhook is checked against X-Twilio-Signature in constant time; a request without a valid signature gets a 403. The answer connects the call to a Media Stream at /v1/voice/twilio/media with a single-use ticket. Audio is µ-law 8 kHz both ways. When the caller talks over the agent, venom sends clear, and Twilio's mark echoes show how much of the reply the caller heard. Closing the stream hangs up.

Outbound calls (POST /v1/calls with "transport": "twilio" and to) come from voice.twilio.from and use /v1/voice/twilio/status for status callbacks. Answering-machine detection is requested for agents with the collections-us pack. A warm transfer updates the live call with <Dial>. When there is a whisper for the person taking the call, Twilio fetches it once from /v1/voice/twilio/whisper/{token}, which plays the clip from GET /v1/voice/clips/{clip}. Both tokens are single use and expire after 2 minutes.

Twilio reaches venom serve through your tunnel, so requests carry the public host name. venom serve accepts that host only on /v1/voice/*.

Asterisk (self-hosted SIP)

Asterisk handles SIP: phones, trunks, registration. venom serve drives it over ARI and exchanges G.711 audio with it over RTP. Nothing reaches a phone network unless you configure a trunk.

  1. Put VENOM_ARI_PASSWORD in .env (any long random value).
  2. Start Asterisk: docker compose --profile sip up -d asterisk. It uses the host's network (Linux, or Docker Desktop with host networking turned on). ARI listens on 127.0.0.1:8088 and SIP on 127.0.0.1:5060.
  3. Add voice.asterisk to venom.yaml and start venom serve with the same .env:
voice:
  asterisk:
    ari_url: http://127.0.0.1:8088
    rtp_ports: 40000-40199   # venom's side; Asterisk's own range is 10000-10199
    codec: pcmu              # or pcma
    trunk: my-trunk          # for outbound calls: PJSIP/<number>@my-trunk

To test, register a softphone on the same machine as user 6001 (password in deploy/asterisk/pjsip.conf, change it) at 127.0.0.1:5060, and dial any number. The dialplan hands every call to venom, and voice.routes picks the agent by the dialled number. A call with no route is hung up.

To reach the phone network, fill in the commented-out my-trunk sections in pjsip.conf with your SIP provider's details. To take calls from other machines, change the transport's bind there and open 5060/udp and Asterisk's RTP range in your firewall yourself. Neither happens by default.

Outbound calls (POST /v1/calls with "transport": "asterisk" and to) originate through the trunk. Agents with the collections-us pack run Asterisk's AMD() first, and the result shows up as the call's answering-machine status. A warm transfer puts the caller on hold music, rings the target, plays the whisper to the target alone, then joins the two and the agent leaves.

The calls API

All routes need the bearer token except the media socket, which needs its ticket. Without voice agents they answer 404 voice_not_configured.

Route What
GET /v1/calls Live and stored calls, newest first; state, agent, limit, before filter them.
POST /v1/calls Start a call: browser, or asterisk/twilio with to.
GET /v1/calls/{id} One call: summary, compliance events, last handoff, per-turn latency metrics.
GET /v1/calls/{id}/events Server-sent events, one per call event; Last-Event-ID or after resumes. An ended call replays its stored events.
GET /v1/calls/{id}/transcript Caller finals and, for the agent, what the caller actually heard.
GET /v1/calls/{id}/recording Stereo 8 kHz WAV (caller left, agent right) when the agent records.
POST /v1/calls/{id}/hangup Hang up a live call.
POST /v1/calls/{id}/transfer Hand a live call to the agent's handoff target.
GET /v1/voice/agents The configured agents, their transports and compliance settings.

The Python client has the same calls: calls, create_call, call, call_events, call_transcript, call_recording, hangup_call, transfer_call and voice_agents.

Where calls are kept

Each call is stored in ~/.venom/calls/<day>/<call id>/: call.json, events.jsonl, transcript.jsonl and recording.wav. Directories are mode 0700 and files 0600. Secrets in transcript and event text are redacted before they are written, as in session logs. Nothing leaves the machine.