Skip to content
Docs menu

The orchestrator's audio API

Generate speech, list voices and transcribe audio through the orchestrator, route it to local or cloud speech models, and price it per character or per second.

View .md

The orchestrator serves OpenAI's audio API with your client key, in front of any speech model it can reach. Any OpenAI SDK works against it unchanged.

Endpoint What it does
POST /v1/audio/speech text to speech, the audio streamed back as it is generated
GET /v1/audio/voices?model= the voices of every speech target of a model or alias
POST /v1/audio/transcriptions speech to text (multipart, OpenAI's form)
POST /v1/audio/translations speech to English text (the same form)

Text to speech

The body is OpenAI's: model, input, voice (an id, or {"id": "..."}), response_format (mp3 by default, opus, aac, flac, wav, pcm), speed (0.25 to 4.0), instructions, and stream_format (audio, or sse for OpenAI's speech.audio.delta events). Venom adds stream and one extension object, venom_options:

Field Meaning
venom_options.language a BCP-47 language hint for providers that take one
venom_options.sample_rate the sample rate in Hz, where the provider lets you pick it
venom_options.provider an object merged into the provider's own request body, for options OpenAI's body has no field for
curl -s http://127.0.0.1:8787/v1/audio/speech \
  -H "Authorization: Bearer $VENOM_API_KEY" -H 'Content-Type: application/json' \
  -d '{"model": "venom-speak", "input": "Build passed.", "voice": "coral"}' -o out.mp3

From the Python SDK, the extension goes in extra_body:

client.audio.speech.create(model="venom-speak", voice="coral", input="Build passed.",
                           extra_body={"venom_options": {"language": "en"}})

The audio is written to you as the provider sends it. The orchestrator never collects the whole answer first. If a target fails before its first byte, the next target of the alias is tried. pcm is raw signed 16-bit mono, and the answer names its rate: Content-Type: audio/pcm;rate=24000;channels=1 and an x-venom-sample-rate header (24 kHz unless the provider says otherwise).

Voices

GET /v1/audio/voices?model=venom-speak returns {"object": "list", "data": [...]}. Each voice has an id, a name, optional languages, gender and preview_url, its provider (the upstream's name) and its model. Voice ids belong to the provider, so pick one from the provider that will serve the request.

Routing speech

Only models marked as speech models are tried for /v1/audio/speech. A model counts as one when it has capabilities.tts: true (on its upstream, or per model in model_info), when its provider's kind is a speech kind, or when its id is a known speech family (tts-1, gpt-4o-mini-tts, Kokoro, Piper, ...). capabilities.stt marks speech-to-text models the same way. /v1/models reports both, and lists a speech alias with endpoints: ["/v1/audio/speech"].

The shipped venom-speak alias speaks through OpenAI's gpt-4o-mini-tts when OPENAI_API_KEY is set. To keep text on your machine, put a local server with OpenAI's speech API first:

upstreams:
  - name: kokoro
    kind: generic
    base_url: http://localhost:8880/v1
    models: [kokoro]
    model_info:
      kokoro: {capabilities: {tts: true}}
routes:
  - alias: venom-speak
    targets:
      - {upstream: kokoro, model: kokoro}
      - {upstream: openai, model: gpt-4o-mini-tts}

Price and usage

Speech is priced by input characters (per_1m_chars, USD per million characters) and transcription by audio duration (per_second, USD per second), in config/prices.yaml or under pricing.models in orchestrator.yaml. The request log records audio_chars and audio_ms per request, and key budgets count both.