Skip to content
Docs menu

Long tasks and resume

Keep hours-long runs alive through crashes, reboots and provider usage limits with checkpoints and venom resume, and let the stuck detector shift gears or stop a run that loops.

View .md

A task that runs for hours should survive a crash, a reboot, a dropped network or a provider's usage limit without losing work. Venom writes a small checkpoint as it goes, venom resume continues from it, and a usage limit with a reset time makes a run wait instead of fail. A separate stuck detector notices a run that goes in circles.

Resume an interrupted task

venom resume                  # the newest interrupted task of this directory
venom resume 01a13            # one task by id or unique prefix, from any directory
venom resume --list           # what is waiting here (--all: every directory)

A bare venom in a directory with an interrupted task prints one line about it and asks Resume it? [Y/n] (terminals only; resume.prompt: false turns that off).

A task is resumable when it is waiting or paused, or running under a process that no longer exists. One running under a live process is not yours to take.

Task kind What resume does
A plain session Reopens its session file. The first message says what was interrupted, which tool calls were in flight ("they may or may not have taken effect; check before repeating") and how the worktree changed since. If the session file is gone it starts fresh with the task's first line.
A plan-execute run Continues at its first step that is not done (same as /plan-exec resume).
A scheduled run Continues it in its worktree, interactively or with -p. Runs the session guard paused also resume from the inbox.
A task parked on a limit Waits for the reset first (status line: limit: resuming in 2h 10m).

venom resume starts the new process with VENOM_RESUME_TASK set to the task id; that process adopts the checkpoint. You don't set it yourself.

Usage limits

When a run ends because the provider answered a limit that names its reset time, the task parks instead of failing. The time comes from response headers (Retry-After, x-ratelimit-reset*, anthropic-ratelimit-*-reset) or from the error text (try again in 2h 15m, resets at 3:00 PM, an ISO timestamp, a Unix time).

  • The checkpoint goes to waiting with waitUntil, and the process sleeps in 30-second slices, so a laptop that slept wakes on time. Escape stops the wait; the task stays resumable.
  • At the reset the run continues with one hidden message. It works the same headless.
  • A 429 that names no time is retried with backoff as usual. A limit no wait fixes (a tokens-per-minute error, a spend budget with no reset) fails as before.
  • resume.fallback_gear switches to that gear at once instead of waiting; the session stays on it.
  • resume.max_wait_hours bounds one wait. A later reset leaves the task parked and resumable.
  • resume.on_limit: fail makes a limit end the run.

Settings

Key Type Default What it does
resume.on_limit wait | fail wait fail: a limit ends the run.
resume.fallback_gear string none Switch to this gear instead of waiting (it must be available).
resume.max_wait_hours number, 0.01 to 168 12 The longest one wait.
resume.prompt boolean true Ask at start whether to resume this directory's interrupted task.
resume.checkpoints boolean true Write checkpoints at all. false also turns off resume and the limit wait.
resume:
  on_limit: wait
  fallback_gear: eco
  max_wait_hours: 6

The checkpoint

One JSON file per task in ~/.venom/tasks/<id>.json ($VENOM_HOME/tasks), rewritten atomically at each turn and each plan step. It holds only what a resume needs:

Field What it holds
kind session, plan-exec or schedule
sessionFile, sessionId, gear the session to reopen and the gear in force
plan a plan-execute run's steps and their state
pending tool calls in flight (anything but reads), written before the call runs
worktree head, branch and changed files, at most every 20 seconds
status, waitUntil, reason running, waiting, paused, failed or done
pid, updatedAt, usd the writing process, when, and the spend so far

An empty session leaves nothing. Checkpoints are never written inside the project. Finished ones are pruned after 7 days and abandoned ones after 30. No checkpoint holds a key or a prompt's text beyond its first line.

A crash loses the model call in flight, not the work: the session file has every completed turn, the checkpoint has the step and the files are on disk. Checkpoints are local to the machine; to move a session to another machine use venom handoff.

The stuck detector

Within a session, the stuck detector watches the main agent (not subagents) for two loops:

  • Oscillation: a file written back to a content it had earlier (A, then B, then A again) more than stuck.oscillations times.
  • The same failing call: one tool call with the same arguments failing with the same error stuck.repeated_failures times, with anything in between. A test command that keeps failing the same way after edit after edit is the usual case.

The first time, venom shifts up one gear (eco to normal to power, or to stuck.gear) and tells the model why. After that it asks you whether to continue. In a headless run the first detection tells the model to try a clearly different approach and posts a Stuck: item to the inbox; a second one stops the run, and the model is told to report what it tried and what blocks it.

Key Type Default What it does
stuck.oscillations integer 0 to 100 2 Returns to an earlier file content before it counts. 0: off.
stuck.repeated_failures integer 0 to 100 3 Same call, same error, before it counts. 0: off.
stuck.escalate boolean true Shift up a gear once before asking.
stuck.gear string next gear The gear to shift to.
stuck:
  oscillations: 3
  repeated_failures: 3
  escalate: true
  gear: power

Headless runs also have a per-run guard: the same call 3 times in a row, 3 bash failures in a row with the same error, or the tool-call ceiling stops the call and posts a Stuck: inbox item with the last 8 calls.