Skip to content
Docs menu

Public mesh hardening

How the public mesh checks that a daemon run by a stranger serves the registered model at the promised quality, what was measured, and what is still weak.

View .md

On the public mesh, anyone can host a daemon and earn per token. This page summarises how venom checks such a daemon and where the checks are still weak. The normative contract is docs/public-mesh.md and the decision record is ADR 0023; where they differ from this summary, they win.

The threat

A public daemon's operator can earn the same money more cheaply by serving a smaller model, a coarser quantisation or a cheaper engine setting (an FP8 KV cache); by editing, truncating or caching prompts and answers; by inflating token counts; by faking latency to attract traffic; or by behaving only when it thinks it is tested. The daemon's code is open, so no check may depend on a secret the daemon can see.

The pipeline

  1. Verifiable sampling (venom-sampler/1). Every public request carries a secret 16-byte nonce. The engine picks each token as the Gumbel-max over the top-k/top-p candidates, with noise from HMAC-SHA256(nonce, position ‖ token). Given the model's log-probabilities, anyone can replay the choice. A vLLM logits-processor plugin implements it.
  2. Receipts. The daemon signs a receipt binding the prompt token hash, the output token ids, the streamed text and the counts.
  3. Audits. An auditor replays sampled jobs on reference hardware and compares the daemon's tokens with the ones the registered model would pick.
  4. Sequential evidence. One audit is tuned to a one-in-a-million false-positive rate, which makes it blind to an artifact that is only slightly wrong. The auditor therefore accumulates evidence across audits (a CUSUM-style statistic), so a small, steady excess, such as a one-step-coarser quantisation, is caught over many jobs.
  5. Witnessed latency. The cloud stamps time to first token on the first streamed line that carries a real output token, so an empty opening chunk sent at once no longer looks fast. A "first-token feint" (streaming a guessed opener before prefill ends, then sampling honestly) is caught by the sequential evidence at its second audited occurrence.
  6. Preflight. A registry entry can carry up to 8 golden cases. At start-up and every 6 hours the daemon runs them through its public-job path, locally, and advertises a model for public traffic only when they pass. The cloud never sees a preflight.
  7. llama.cpp without a plugin. A shim (venom-llamacpp) puts llama.cpp behind the sampler contract so it can be audited too.

Known shortfalls

  • Verdict results remain visible to the daemon after each epoch, so a daemon learns that it failed an audit, though not how close it is to the alarm.
  • A non-streamed job's time to first token is its whole duration; a stream padded with over 64 KiB of tokenless data falls back to the done time and counts as an SLO miss, not as abuse.
  • The preflight's thresholds come from a toy model and have not been run against real vLLM or llama.cpp.
  • The llama.cpp path replays every prompt position, which is slow on long prompts, and approximates the rank of rare prompt tokens.
  • Region verification, an attested trust class, payouts and KYC are out of scope.

Reproducing

In a venom checkout, the adversarial toy suite is go test ./internal/audit -run Adversarial -v in services/go-orchestrator, and the shared vectors run in go test ./internal/provenance -run Cusum and make test-cloud. make public-mesh-e2e runs the whole public mesh on loopback.