---
title: Kubernetes
description: Run the orchestrator on Kubernetes from venom's kustomize manifests, with keys in Secrets and the config as a generated ConfigMap, optionally as a Venom Cloud daemon.
section: deploy
order: 40
audience: [operator]
sources: [docs/kubernetes.md, deploy/kubernetes, docs/adr/0037-kubernetes-manifests.md]
features: [deploy.kubernetes]
---

`deploy/kubernetes/` in a venom checkout is a kustomize base (the orchestrator) and one overlay (the same orchestrator as a Venom Cloud daemon). These are plain manifests, not an operator or a Helm chart (ADR 0037): the orchestrator is one SQLite-backed process with one config file and a few secrets, which is what a Deployment already describes. kustomize is built into `kubectl`, so there is nothing else to install, and nothing in the manifests calls out anywhere.

To run Venom Cloud, its scheduler and Postgres as well, use the [Helm chart](/docs/deploy/kubernetes-helm.md). This page covers the orchestrator alone.

## Install {#install}

The orchestrator image is a release artifact (`venom/orchestrator:<version>`, built locally by `make release`, [Releases and signatures](/docs/deploy/releases.md)). Push it to a registry your cluster can pull from, then:

```bash
kubectl create namespace venom
kubectl -n venom create secret generic venom-orchestrator-keys \
  --from-literal=VENOM_API_KEY="$(openssl rand -hex 24)" \
  --from-literal=VENOM_ADMIN_TOKEN="$(openssl rand -hex 24)"
kubectl -n venom create secret generic venom-provider-keys --from-env-file=provider-keys.env   # OPENAI_API_KEY=...

cd deploy/kubernetes/base
kustomize edit set image venom/orchestrator=registry.example.com/venom/orchestrator:<version>
cd ../../..
kubectl kustomize --load-restrictor=LoadRestrictionsNone deploy/kubernetes/base | kubectl -n venom apply -f -
kubectl -n venom rollout status deploy/venom-orchestrator
```

The load restrictor is off because the ConfigMap is generated from `config/orchestrator.yaml` and `config/prices.yaml`, outside the kustomization directory. Without `kustomize edit`, the base uses the tag `latest`.

Clients inside the cluster use `http://venom-orchestrator.venom.svc:8787/v1` with the `VENOM_API_KEY` you created.

## What gets created {#objects}

| Object | What |
|---|---|
| `Deployment` `venom-orchestrator` | One replica, strategy `Recreate`. Non-root (uid 100, gid 101, the image's `venom` user), read-only root filesystem, every capability dropped, `RuntimeDefault` seccomp, no service account token. Requests 100m CPU and 128 Mi, limit 512 Mi. An `emptyDir` at `/tmp`. |
| `Service` `venom-orchestrator` | `ClusterIP` on 8787: `/v1/*`, `/admin/v1/*`, `/metrics`, `/healthz`, `/readyz`. Nothing is exposed outside the cluster. |
| `ConfigMap` `venom-orchestrator-config-<hash>` | `orchestrator.yaml` and `prices.yaml`, mounted read-only at `/app/config`. A changed file changes the name hash and rolls the pod. `${VAR}` in the file expands from the container environment, as in compose. |
| `PersistentVolumeClaim` `venom-orchestrator-data` | 5 Gi, ReadWriteOnce, at `/data`: keys, usage, sessions and the mesh credential. |
| `ServiceAccount` `venom-orchestrator` | No RBAC; the pod does not talk to the Kubernetes API. |

**Probes.** Startup (every 2 s, up to 30 tries) and liveness (every 10 s) on `/healthz`. Readiness on `/readyz`, which answers 503 until at least one upstream is healthy, so a pod with no reachable inference backend gets no traffic.

**Why one replica.** The stores are SQLite on a ReadWriteOnce volume; two writers would corrupt them. More capacity means more orchestrator instances (each its own release of the manifests), not more replicas. The orchestrator migrates its own database at start, so an upgrade is a new image tag.

## Keys and config {#secrets}

Keys come from Secrets only; the manifests hold none.

| Secret | Keys | Required |
|---|---|---|
| `venom-orchestrator-keys` | `VENOM_API_KEY`, `VENOM_ADMIN_TOKEN` | yes. Without a client key the orchestrator refuses to start on a non-loopback address. |
| `venom-provider-keys` | any: each key becomes an environment variable (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `OLLAMA_BASE_URL`, `VLLM_BASE_URL`, …) | no. An upstream whose key is empty is disabled. |
| `venom-mesh` | `join-token` | only for the mesh overlay. |

`deploy/kubernetes/secrets.example.yaml` is a template with placeholder values and is part of no kustomization; prefer the `kubectl create secret` commands above, which never write a key to a file.

To change upstreams, point `orchestrator.yaml=` in `base/kustomization.yaml` at your own copy. In-cluster model servers are plain environment variables: put `OLLAMA_BASE_URL=http://ollama.models.svc:11434/v1` (for example) in `venom-provider-keys` or in a patch.

## Exposing it {#ingress}

Add your own Ingress or Gateway route to the `venom-orchestrator` Service, with TLS, and only after the keys above are real. The admin API is on the same port, so consider a route for `/v1` only and reach `/admin/v1` with `kubectl port-forward`:

```bash
kubectl -n venom port-forward svc/venom-orchestrator 8787:8787
curl -s -H "Authorization: Bearer $VENOM_ADMIN_TOKEN" http://127.0.0.1:8787/admin/v1/upstreams
```

## As a Venom Cloud daemon {#mesh}

```bash
kubectl -n venom create secret generic venom-mesh --from-file=join-token=/dev/stdin   # paste the vjt_ token, then Ctrl-D
# set VENOM_CLOUD_URL in deploy/kubernetes/overlays/mesh/mesh-env.yaml
kubectl kustomize --load-restrictor=LoadRestrictionsNone deploy/kubernetes/overlays/mesh | kubectl -n venom apply -f -
```

The overlay adds `VENOM_CLOUD_URL` (a placeholder, `https://cloud.example.com`, that you edit), `VENOM_MESH_NAME` (the pod name) and `VENOM_MESH_JOIN_TOKEN` (from the optional `venom-mesh` Secret). The daemon dials out to the cloud and needs no inbound route. It enrolls once and keeps its credential on the data volume, so after the first start you can delete the `join-token` key. Provider keys stay in the cluster. The base never sets `VENOM_CLOUD_URL`, so mesh mode exists only where the overlay is applied.

## Troubleshooting {#troubleshooting}

| Symptom | Cause and fix |
|---|---|
| Pod never becomes ready | `/readyz` is 503 until one upstream is healthy. Check `GET /admin/v1/upstreams` through a port-forward, and that model server URLs resolve from the pod. |
| `CreateContainerConfigError` | The `venom-orchestrator-keys` Secret is missing in the namespace. |
| `kubectl kustomize` fails with a load restriction error | Add `--load-restrictor=LoadRestrictionsNone`. |
| Second pod stuck in `ContainerCreating` | Something raised `replicas`; the volume is ReadWriteOnce. Keep one replica. |

## Related

- [Docker Compose and profiles](/docs/deploy/compose.md): the same orchestrator with Docker.
- [Orchestrator environment variables](/docs/orchestrator/environment.md) and [configuration](/docs/orchestrator/configuration.md).
- [Observability](/docs/deploy/observability.md): scraping `/metrics`.
