Skip to content
Docs menu

Evidence bundles

Print the proof a change carries with venom evidence, as JSON or a pull request comment, and build it from code with @venom/harness/evidence.

View .md

A reviewer should not have to guess whether a change works. venom evidence prints the proof the change carries, as JSON (schema id venom.evidence/v1) or as markdown short enough for one pull request comment.

venom evidence                      # uncommitted changes against HEAD
venom evidence --base main --json   # a whole branch, as JSON
venom evidence --base main --markdown --extra proof.json --post 123

What is in it

Gathered from the repository, offline:

  • the changed files (staged, unstaged, untracked) and the test files the change adds;
  • the tests that cover the diff, from venom's impacted-test selection;
  • with --coverage <file> (a Go cover profile or lcov; repeatable): coverage of the changed lines and the added lines no test reaches. A file the reports do not mention is "not measured", never "uncovered";
  • invariants read off the diff: no key-shaped strings, no conflict markers, a test change next to every source change.

Handed in with --extra <file.json>, because only the caller knows them: runs (test commands with results and counts), bench and evals (before and after pairs, with better: "lower" | "higher"), extra invariants, a session (an orchestrator session id and a replay link), screenshots (paths), and an optional risk (score, level, reasons).

Nothing runs a test and nothing calls a model. The one network use is --post <pr>, which hands the markdown to gh pr comment.

Eval verdict

A harness, prompt or model change can carry an eval verdict in evals. In a venom checkout, make eval-verdict BASE=main OUT=verdict.json runs only the eval tasks the diff touches, at k=3 under a $1 cap, compares them with the committed baseline and writes the result for --extra. It fails on a regression, a safety drop, no paired task, or a run over $1.

From code

import { buildEvidence, renderEvidenceMarkdown } from "@venom/harness/evidence";

const bundle = await buildEvidence({ cwd: process.cwd(), base: "main", runs: [] });
console.log(renderEvidenceMarkdown(bundle));

venom ship (or /ship in a session) builds this bundle for the branch, posts it on the pull request it opens and hands that pull request to the gate.