---
title: Chat, responses, embeddings and models
description: Call chat completions, completions, the Responses API (including background responses) and embeddings through the orchestrator, and read what the model list tells you.
section: orchestrator
order: 60
audience: [builder, agent]
sources: [docs/architecture.md, services/go-orchestrator/AGENTS.md, services/go-orchestrator/internal/server/server.go, services/go-orchestrator/internal/server/prewarm_models.go]
features: [orchestrator.api-chat, orchestrator.api-completions, orchestrator.api-embeddings, orchestrator.api-responses, orchestrator.api-models, orchestrator.api-prewarm]
---

These are the OpenAI endpoints most clients use. Requests and answers are OpenAI's; the orchestrator rewrites `model`, applies each upstream kind's parameter rules and relays the answer, streamed or not. The examples assume `export OPENAI_BASE_URL=http://127.0.0.1:8787/v1` and a client key in `OPENAI_API_KEY`, which the OpenAI SDKs read.

## Chat completions {#chat}

`POST /v1/chat/completions`, streaming (SSE) and non-streaming, with tools, images and reasoning.

```bash
curl -N "$OPENAI_BASE_URL/chat/completions" -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" -H "x-venom-session: demo-1" \
  -d '{"model":"venom-default","stream":true,"messages":[{"role":"user","content":"Write a haiku about Go"}]}'
```

What the orchestrator does on the way:

- Removes parameters the target doesn't take and renames the ones it spells differently (`max_completion_tokens` → `max_tokens` on Ollama, `seed` → `random_seed` on Mistral), and maps `reasoning_effort` to the nearest level the model accepts.
- Normalises reasoning to `reasoning_content` in answers, and sends earlier turns' reasoning back in the field the server reads (or drops it where it reads none).
- Adds `stream_options.include_usage` where needed so a stream ends with usage; a client that didn't ask for usage doesn't get the extra chunk.
- Repairs malformed tool-call arguments some local models emit (a single-quoted or trailing-comma JSON object), counted in `venom_tool_call_repairs_total`.
- A request with an image skips targets known to be text-only, or goes to the route's `images` route.

A stream ends at its terminal event (`data: [DONE]`); if an upstream fails after the first byte, the stream ends with an error event in OpenAI's shape and the client retries.

## Completions and embeddings {#completions}

`POST /v1/completions` and `POST /v1/embeddings` are routed the same way and passed through after the model is rewritten. On a `mistral` target, `/v1/completions` with a `suffix` becomes Codestral's fill-in-the-middle call.

```python
from openai import OpenAI

client = OpenAI()  # OPENAI_BASE_URL and OPENAI_API_KEY from the environment
vectors = client.embeddings.create(model="venom-embed", input=["def add(a, b):", "class User:"])
print(len(vectors.data[0].embedding))
```

Embeddings take `input` as a string or an array, `dimensions` and extension fields such as `input_type`. `encoding_format: "base64"` (the SDKs' default) works against every kind: where the upstream answers with floats, the orchestrator encodes them as OpenAI does. Bedrock serves Titan Text Embeddings V2 (one input per request) and Cohere Embed. The shipped `venom-embed` alias has only a local Ollama target, so code never leaves the machine for an embedding.

## Responses API {#responses}

`POST /v1/responses`, streaming and non-streaming. To a target whose `capabilities.responses` is true (OpenAI by default) it is passed through. To every other target it is translated to chat completions and the answer translated back into a Response object or Responses SSE events (`response.created`, `response.output_text.delta`, `response.function_call_arguments.delta`, …, `response.completed`), so a Responses client works against Ollama or vLLM off the grid. Translated answers carry `x-venom-translated: chat_completions`.

```ts
import OpenAI from "openai";

const client = new OpenAI(); // OPENAI_BASE_URL and OPENAI_API_KEY from the environment
const res = await client.responses.create({ model: "venom-default", input: "Name three prime numbers" });
console.log(res.output_text);
```

Translated: `input` (text, messages, function calls and outputs, reasoning items, `input_image` URLs), `instructions`, function `tools`, `tool_choice`, `max_output_tokens`, `temperature`, `top_p`, `parallel_tool_calls`, `text.format`, `reasoning.effort`, `user`, `safety_identifier`, `prompt_cache_key`, `service_tier`; `metadata` is echoed but not sent. The translation is stateless: `previous_response_id` and `conversation` work only on native targets, and with none left the answer is `400 stateful_request_unsupported`. Hosted tools (`web_search`, `file_search`), `item_reference` inputs, `input_file` parts and stored prompts are `400 unsupported_request` unless a native target remains.

### Background responses {#background}

`background: true` returns at once with a queued Response and runs the request on its own.

| Endpoint | What it does |
|---|---|
| `GET /v1/responses/{id}` | The response this key created. On a native target it is read from the provider; on a translated one it is the stored object, and `?stream=true` replays its events (after `starting_after=<sequence_number>`) and follows a running one. |
| `POST /v1/responses/{id}/cancel` | Cancels it; a completed translated one is `400 invalid_state`. |
| `DELETE /v1/responses/{id}` | Cancels a running one and deletes it. |

Another key's id, or an unknown one, is `404 not_found`. A native background response is billed once, at the usage the provider reports when it ends (polled for up to 6 hours); a translated one is billed when its run ends, and its final Response is kept for 24 hours.

## Models {#models}

`GET /v1/models` lists every route alias plus the concrete `upstream/model` ids that can be served; `GET /v1/models/{id}` is one entry (the `/` of an `upstream/model` id may be sent raw or as `%2F`; unknown is `404 model_not_found`). A key with `allowed_models` sees only those. Entries carry non-standard fields clients can use:

| Field | Meaning |
|---|---|
| `context_window` | Tokens a request may use. For an alias, the smallest window among the targets that can serve now, so it follows upstream health. |
| `capabilities` | `vision` (true only when every reachable target takes images), and `tts` / `stt` on speech models. |
| `max_inflight` | How many requests to this id the orchestrator runs at once (omitted when unbounded). |
| `endpoints` | Set on models that serve something other than chat: `["/v1/embeddings"]`, `["/v1/rerank"]`, `["/v1/systemone"]`, `["/v1/audio/transcriptions"]`, `["/v1/audio/speech"]`, `["/v1/images/generations"]`. Chat clients skip these. |
| `unavailable` | On an alias, the targets skipped because their upstream lacks the model, each with `reason` and `fix`. |
| `cloud_targets` | On an alias, the enabled targets that are not local: where prompts can go. |

The list object also has a top-level `profiles` array with the configured route profile names.

## Prewarm {#prewarm}

`POST /v1/prewarm` loads the local models a set of ids routes to before the first turn, so a session doesn't wait for a cold load. The venom CLI sends it when a session starts on a gear or switches gear.

```bash
curl -s "$OPENAI_BASE_URL/prewarm" -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" -d '{"models":["venom-default","venom-fast"],"keep_alive_s":3600}'
```

Each id (1 to 8, else `400 invalid_body`) is resolved to the first target its route would use. Ollama targets are loaded only when they fit beside what is loaded, so a prewarm never evicts a model; the answer lists each with `state` `loaded`, `warm` (already loaded, keep-alive renewed), `skipped` (doesn't fit, with the figures), `not_local` (a cloud target) or `not_served`. `keep_alive_s` defaults to 1 hour, at most 12.

## Request headers {#request-headers}

| Header | What it does |
|---|---|
| `x-venom-session` | Keeps a session on one target for cache reuse; `<session>:<kind>` marks a side request. |
| `x-venom-parent-session` | Rolls a subagent's spend up under its parent; also inherits its experiment variant. |
| `x-venom-profile` | Picks a route profile. |
| `x-venom-role`, `x-venom-gear` | The spend lane (`workers`, `smart`, `vision`, `browser`, `classifier`, `summary`, `reflex`, …) and gear, recorded in the request log. |
| `x-venom-intent` | Meters the request against an intent's spend cap ([Keys, budgets and limits](/docs/orchestrator/keys-and-budgets.md#intents)). |
| `x-venom-service-tier`, `x-venom-urgency` | Provider tier and batch requests ([Batches, files and service tiers](/docs/orchestrator/batches-and-files.md#service-tiers)). |
| `x-venom-cache-ttl` | `long` asks for the model's longer prompt-cache lifetime (`cache_retention`). |
| `x-venom-dedup` | `off` opts out of request deduplication. |
| `Idempotency-Key` | Collapses retries of the same request and replays the completed answer. |

## Response headers {#response-headers}

Every answer carries `x-venom-request-id` and `x-request-id` (the upstream's own, when it sent one). Committed answers carry `x-venom-upstream` and `x-venom-model` before the first byte. Depending on the request: `x-venom-fallback`, `x-venom-base-model`, `x-venom-translated`, `x-venom-variant`, `x-venom-profile`, `x-venom-response-cache`, `x-venom-dedup`, `x-venom-queued-ms`, `x-venom-budget-warning`, `x-venom-guardrail`, `x-venom-context-cost-usd`. Upstream headers pass by allowlist only (content type and length, request ids, `retry-after`, `x-ratelimit-*`, `openai-processing-ms`, `openai-version`); cookies, account ids and server banners never reach the client.

## Related

- [Routes and strategies](/docs/orchestrator/routing.md) for fallback reasons, [Keys, budgets and limits](/docs/orchestrator/keys-and-budgets.md), [Batches, files and service tiers](/docs/orchestrator/batches-and-files.md).
