Rerank, images and decisions
Rerank documents, generate images and ask typed decision questions through the orchestrator's routing, keys and budgets.
Three endpoints that are not chat but go through the same routing (aliases, fallback, breakers), client-key auth, rate limits, budgets and request log. Never streamed.
Rerank
POST /v1/rerank orders documents by relevance to a query, in the Cohere / Jina shape. The shipped venom-rerank alias uses Model Studio's qwen3-rerank once QWEN_API_KEY is set; put a local reranker first to keep documents on the machine.
curl -s http://127.0.0.1:8787/v1/rerank -H "Authorization: Bearer $VENOM_API_KEY" -H "Content-Type: application/json" \
-d '{"model":"venom-rerank","query":"how do I rotate a key?","documents":["Keys are minted with POST /admin/v1/keys.","The console has a dark theme."],"top_n":1,"return_documents":true}'{"id": "…", "object": "list", "model": "qwen/qwen3-rerank",
"results": [{"index": 0, "relevance_score": 0.91, "document": {"text": "Keys are minted with POST /admin/v1/keys."}}],
"usage": {"prompt_tokens": 31, "total_tokens": 31}}| Field | What it does |
|---|---|
model |
An alias or upstream/model. |
query |
Required. |
documents |
1 to 1000, strings or {"text": …}. |
top_n (or Voyage's top_k) |
Cut the results to this many, best first. |
return_documents |
Include each document's text (from the request; an upstream is never asked to echo them). |
max_tokens_per_doc, instruct |
Passed to rerankers that take them. |
Each upstream gets the request in its own dialect (rerank: on the upstream, else by kind): jina (POST {base_url}/rerank: vLLM, llama.cpp, SGLang, Jina, Together; the default), cohere, voyage, tei (Hugging Face text-embeddings-inference), dashscope (Model Studio's native text-rerank), and Bedrock's cohere.rerank-v3-5:0 and amazon.rerank-v1:0 through InvokeModel. Cost is tokens in at the table's input rate plus search_unit per search; a reranker that reports no tokens is logged at an estimate. Errors: 400 invalid_rerank_request (no query, no documents or over 1000, top_n < 1, a document of the wrong shape); a reranker's own error is relayed in OpenAI's shape.
docker run -p 8081:80 ghcr.io/huggingface/text-embeddings-inference:cpu-1.8 --model-id BAAI/bge-reranker-v2-m3upstreams:
- {name: tei, kind: generic, rerank: tei, base_url: http://localhost:8081/v1, local: true}
routes:
- alias: venom-rerank
targets:
- {upstream: tei, model: BAAI/bge-reranker-v2-m3}
- {upstream: qwen, model: qwen3-rerank}Images
POST /v1/images/generations is OpenAI's image API: {model, prompt, n (1–10), size ("WxH" or "auto"), response_format ("url" or "b64_json")} in, {created, data: [{url | b64_json, revised_prompt}], usage} out.
upstreams:
- name: sd
kind: sdapi
base_url: http://localhost:7860 # the server root, no /v1
routes:
- alias: venom-image
targets:
- {upstream: sd, model: default} # "default" keeps the server's loaded checkpointcurl -s http://127.0.0.1:8787/v1/images/generations -H "Authorization: Bearer $VENOM_API_KEY" \
-H "Content-Type: application/json" -d '{"model":"venom-image","prompt":"a lighthouse at dusk","size":"768x512"}'A target is a kind sdapi backend (AUTOMATIC1111, Forge, SD.Next, stable-diffusion.cpp's server: POST /sdapi/v1/txt2img, which also takes negative_prompt, steps, seed, cfg_scale and sampler_name) or an OpenAI-compatible one ({base_url}/images/generations: OpenAI, LocalAI). Backends are always asked for base64; the default url is a signed link into the orchestrator's own file store (/v1/files/{id}/content?exp=&sig=) that works without a key for an hour, and the file is kept 24 hours. Errors: 400 missing_prompt, missing_model, invalid_n, invalid_response_format or a bad size; 403 model_not_allowed; 404 model_not_found; a backend's own 4xx relayed; 502 upstream_error when every target failed.
Decisions
POST /v1/systemone asks typed questions about a state in TypeSafe's System One wire format. The venom CLI uses it (alias venom-decide) for advisory decisions such as permission triage; decisions are never the final authority.
curl -s http://127.0.0.1:8787/v1/systemone -H "Authorization: Bearer $VENOM_API_KEY" -H "Content-Type: application/json" \
-d '{"model":"venom-decide","state":"The user asked to delete the build directory.","questions":{"safe":{"type":"noul","instructions":"Is this safe to run without asking?"}}}'The request is {"state": string | object | array, "model", "questions": {"<id>": {"type": "noul" | "choice" | "score", "instructions", "criteria"}}}; the upstream's answer ({"model", "answers": {"<id>": …}, "usage"}) comes back unchanged with x-venom-upstream and x-venom-model. Only decision models serve it (kind typesafe, or any kind with capabilities.decisions: true, such as an open-weights decision server on kind generic placed before TypeSafe), and they serve nothing else.
| Status | Code | Cause |
|---|---|---|
| 400 | invalid_decision_request |
no state, questions not a non-empty object, or stream: true |
| 400 | unsupported_endpoint |
the model is a chat model (or a chat endpoint was asked of a decision model) |
| 413 | request_too_large |
over 2 MiB (or max_body_bytes if smaller) |
| 422 | TypeSafe's validation error, relayed | |
| 502 | upstream_auth_failed |
TypeSafe refused the orchestrator's key |
| 503 | no_upstream |
TYPESAFE_API_KEY is not set |
429 and 529 from TypeSafe are retried with backoff, then relayed with their retry-after. Without TYPESAFE_API_KEY the alias is not listed in /v1/models.
Related
- Upstreams and kinds for
rerank:,sdapiandtypesafe, Batches, files and service tiers for the file store.