Skip to content
Docs menu

Rerank, images and decisions

Rerank documents, generate images and ask typed decision questions through the orchestrator's routing, keys and budgets.

View .md

Three endpoints that are not chat but go through the same routing (aliases, fallback, breakers), client-key auth, rate limits, budgets and request log. Never streamed.

Rerank

POST /v1/rerank orders documents by relevance to a query, in the Cohere / Jina shape. The shipped venom-rerank alias uses Model Studio's qwen3-rerank once QWEN_API_KEY is set; put a local reranker first to keep documents on the machine.

curl -s http://127.0.0.1:8787/v1/rerank -H "Authorization: Bearer $VENOM_API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"venom-rerank","query":"how do I rotate a key?","documents":["Keys are minted with POST /admin/v1/keys.","The console has a dark theme."],"top_n":1,"return_documents":true}'
{"id": "…", "object": "list", "model": "qwen/qwen3-rerank",
 "results": [{"index": 0, "relevance_score": 0.91, "document": {"text": "Keys are minted with POST /admin/v1/keys."}}],
 "usage": {"prompt_tokens": 31, "total_tokens": 31}}
Field What it does
model An alias or upstream/model.
query Required.
documents 1 to 1000, strings or {"text": …}.
top_n (or Voyage's top_k) Cut the results to this many, best first.
return_documents Include each document's text (from the request; an upstream is never asked to echo them).
max_tokens_per_doc, instruct Passed to rerankers that take them.

Each upstream gets the request in its own dialect (rerank: on the upstream, else by kind): jina (POST {base_url}/rerank: vLLM, llama.cpp, SGLang, Jina, Together; the default), cohere, voyage, tei (Hugging Face text-embeddings-inference), dashscope (Model Studio's native text-rerank), and Bedrock's cohere.rerank-v3-5:0 and amazon.rerank-v1:0 through InvokeModel. Cost is tokens in at the table's input rate plus search_unit per search; a reranker that reports no tokens is logged at an estimate. Errors: 400 invalid_rerank_request (no query, no documents or over 1000, top_n < 1, a document of the wrong shape); a reranker's own error is relayed in OpenAI's shape.

docker run -p 8081:80 ghcr.io/huggingface/text-embeddings-inference:cpu-1.8 --model-id BAAI/bge-reranker-v2-m3
upstreams:
  - {name: tei, kind: generic, rerank: tei, base_url: http://localhost:8081/v1, local: true}
routes:
  - alias: venom-rerank
    targets:
      - {upstream: tei, model: BAAI/bge-reranker-v2-m3}
      - {upstream: qwen, model: qwen3-rerank}

Images

POST /v1/images/generations is OpenAI's image API: {model, prompt, n (1–10), size ("WxH" or "auto"), response_format ("url" or "b64_json")} in, {created, data: [{url | b64_json, revised_prompt}], usage} out.

upstreams:
  - name: sd
    kind: sdapi
    base_url: http://localhost:7860        # the server root, no /v1
routes:
  - alias: venom-image
    targets:
      - {upstream: sd, model: default}     # "default" keeps the server's loaded checkpoint
curl -s http://127.0.0.1:8787/v1/images/generations -H "Authorization: Bearer $VENOM_API_KEY" \
  -H "Content-Type: application/json" -d '{"model":"venom-image","prompt":"a lighthouse at dusk","size":"768x512"}'

A target is a kind sdapi backend (AUTOMATIC1111, Forge, SD.Next, stable-diffusion.cpp's server: POST /sdapi/v1/txt2img, which also takes negative_prompt, steps, seed, cfg_scale and sampler_name) or an OpenAI-compatible one ({base_url}/images/generations: OpenAI, LocalAI). Backends are always asked for base64; the default url is a signed link into the orchestrator's own file store (/v1/files/{id}/content?exp=&sig=) that works without a key for an hour, and the file is kept 24 hours. Errors: 400 missing_prompt, missing_model, invalid_n, invalid_response_format or a bad size; 403 model_not_allowed; 404 model_not_found; a backend's own 4xx relayed; 502 upstream_error when every target failed.

Decisions

POST /v1/systemone asks typed questions about a state in TypeSafe's System One wire format. The venom CLI uses it (alias venom-decide) for advisory decisions such as permission triage; decisions are never the final authority.

curl -s http://127.0.0.1:8787/v1/systemone -H "Authorization: Bearer $VENOM_API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"venom-decide","state":"The user asked to delete the build directory.","questions":{"safe":{"type":"noul","instructions":"Is this safe to run without asking?"}}}'

The request is {"state": string | object | array, "model", "questions": {"<id>": {"type": "noul" | "choice" | "score", "instructions", "criteria"}}}; the upstream's answer ({"model", "answers": {"<id>": …}, "usage"}) comes back unchanged with x-venom-upstream and x-venom-model. Only decision models serve it (kind typesafe, or any kind with capabilities.decisions: true, such as an open-weights decision server on kind generic placed before TypeSafe), and they serve nothing else.

Status Code Cause
400 invalid_decision_request no state, questions not a non-empty object, or stream: true
400 unsupported_endpoint the model is a chat model (or a chat endpoint was asked of a decision model)
413 request_too_large over 2 MiB (or max_body_bytes if smaller)
422 TypeSafe's validation error, relayed
502 upstream_auth_failed TypeSafe refused the orchestrator's key
503 no_upstream TYPESAFE_API_KEY is not set

429 and 529 from TypeSafe are retried with backoff, then relayed with their retry-after. Without TYPESAFE_API_KEY the alias is not listed in /v1/models.