Skip to content
Docs menu

Voice

Speak to venom and have it speak back, through your own speech models or cloud ones, behind one OpenAI-shaped audio API.

View .md

Venom's voice features all go through the orchestrator's audio API, so every speech model you can reach (a Kokoro or Whisper server on your machine, vLLM, OpenAI, or a cloud speech provider) is used the same way. Aliases, fallback, budgets, pricing and the request log work for speech as they do for chat.

  • Text to speech: POST /v1/audio/speech, streamed as it is generated.
  • The voices a model speaks in: GET /v1/audio/voices?model=.
  • Speech to text and speech to English: POST /v1/audio/transcriptions and POST /v1/audio/translations.
  • Streaming speech to text over a WebSocket: Realtime transcription.

Providers that don't speak OpenAI's audio API plug in as voice adapters in the orchestrator, one per wire format, so clients never see the difference.

Realtime voice agents, such as a receptionist or a collections agent that answers the phone, are built on the harness with @venom/voice: Voice agents: the core library.

You want Page
dictation and read-aloud in the terminal (/voice, /speak) Voice in the venom CLI
the audio API itself: endpoints, options and voices The orchestrator's audio API
speech from your own Node app, or the agent's speak and transcribe tools Voice in the harness
a realtime voice agent on the harness Voice agents: the core library

In this section

  • Voice in the venom CLI: Dictate into the editor with /voice or ctrl+g r, and have answers read aloud with /speak or ctrl+g k.
  • The orchestrator's audio API: Generate speech, list voices and transcribe audio through the orchestrator, route it to local or cloud speech models, and price it per character or per second.
  • Voice agents: the core library: The @venom/voice package, the call pipeline under venom's AI phone agents, and how to build and test it.
  • Voice in the harness: The @venom/harness/voice client (speak, listVoices, transcribe, translate), the agent's speak and transcribe tools, and venom.yaml's voice keys.
  • Local speech models: Run CosyVoice, Qwen3-TTS, IndexTTS, Spark-TTS, Fish Speech and other open speech models on your own hardware and use them through the orchestrator's OpenAI-shaped audio API, voice cloning included.
  • Realtime transcription: Stream audio to the orchestrator over a WebSocket and get transcripts back as you speak, with OpenAI's realtime transcription events, in front of local or cloud speech models.
  • ElevenLabs, Fish Audio and BytePlus Seed Speech: Route text to speech and transcription to ElevenLabs, Fish Audio or BytePlus Seed Speech through the orchestrator's OpenAI-shaped audio API.
  • Qwen, MiniMax and Mistral speech: Route speech to Qwen TTS on Model Studio and MiniMax T2A, and transcription to Mistral's Voxtral, through the orchestrator's audio API.
  • Voice agents: calls in venom serve: Configure voice agents in venom.yaml, take calls on the browser test line, and read each call's events, transcript and recording over the calls API.