Skip to main content
Experimental Feature: Verify is experimental. The configuration and behavior may change or be removed in future versions. Discuss feedback in GitHub Discussions.

​
What is Verify?

Verify puts a quality gate between your agent’s final output and delivery. After the run finishes, a judge evaluates the output against your criteria. If it fails, the judge’s critique is injected back into the same session as a revision instruction and the agent redoes the output, with all its tool results and completed work still in context, up to maxRedos times. Only then does the output ship to channels, a delegating manager, or the CLI. The redo is a continuation, not a re-run: the agent sees everything it already did (including side effects like sent messages or filed issues), so it revises the deliverable instead of repeating actions.

​
Quick Start

Or with your own rubric:

​
Configuration Options

verify: true uses a generic task-fulfillment rubric. verify: "<string>" is shorthand for { criteria: "<string>" }.

​
Gate Placement (approval agents)

For agents with an approval gate, verify defaults to at: gate: the judge reviews each await_human request before it suspends to the human.
  • Fail → the run does not suspend. The critique returns as a rejection-with-comment tool result, the same protocol as a human rejection, so the agent revises its draft and requests approval again. The human never sees the failed draft.
  • Pass → the gate suspends to the human as normal.
  • Fail-open → after maxRedos pre-review rejections, or on any judge error, the request goes to the human regardless, with the unresolved critique in the session log. The judge protects your attention; it can never deadlock a run or replace your decision.
The judge receives the full approval payload (draft, exact on-approval content, target reference/excerpt, context), so write criteria against what the reviewer would see. Because the check runs before anything posts, this placement is the quality gate; at: output is a post-run audit. Gate placement also works for delegated sub-agents: a leaf’s gate is judged in the leaf, before the request cascades to the root.
The payload is all the built-in judge sees. It runs without tools, so a criterion about a file on disk is really judged against the agent’s account of that file. A payload that paraphrases its own artifact gets rejected even when the artifact is correct, and a payload that flatters a broken artifact passes silently. When the criteria describe something on disk, use a judge agent that can open it.

​
Using a Judge Agent

When a one-line rubric isn’t enough, the evaluation needs its own files, tools, or a stronger model, point judge at another agent file:
The path resolves relative to the agent file that declares it. judge and criteria are mutually exclusive, and model is invalid alongside judge (the judge agent declares its own). The judge’s body is the rubric. Where the built-in judge gets your criteria, an agent judge is handed Apply the evaluation standard defined in your own instructions. in that slot, so the standard has to live in the judge agent itself. Everything else it needs, the task and the candidate output, arrives around it. The judge is otherwise a normal agent: its own frontmatter controls its model, reasoning effort, and tools. That is the point of using one. Grant it read access to the rubric files, exemplars, or the artifact under review, and it evaluates what is actually on disk instead of what the payload claims. It runs as an inspectable child session, so which files it opened and why it decided are visible in the session log, not just the one-line verdict. It records its decision by calling submit_verdict, a tool injected into every judge agent, exactly once:
The tool call is the verdict; a judge should not also write a JSON object in its text. (A trailing JSON object is still parsed as a fallback, which is how the built-in judge reports, but new judge agents should call the tool.) When pass is false, critique is required and must be actionable in one revision, it is injected verbatim into the redo turn. Everything else the judge writes stays in its own reasoning. Judges conventionally stay read-only (they evaluate, the parent revises), but this is not enforced, a judge that runs your test suite is a legitimate design. Judges cannot suspend on approval gates, so never give a judge agent approval: true; a suspended judge counts as a judge error. A nested verify: on a judge is ignored.

​
What Happens on Failure

  1. The verdict and critique are recorded in the session log (visible in the web UI).
  2. The critique is injected as a revision instruction: attempt count, criteria, the critique verbatim, and directives (revise, don’t argue with the review, completed side effects stand, tools allowed for new information).
  3. The agent’s revised output is judged again, up to maxRedos.
  4. If every attempt fails, the last output still ships, and the session is marked verification: failed, a needs-attention signal for you or a delegating manager, not a crash.
A judge that itself errors (model/auth/parse failure) never blocks the run: the output ships unverified with a visible error marker.

​
Cost and Behavior Notes

  • Each verification is one extra LLM call; each redo is a continuation of the existing session (prompt caching keeps it near generation-only cost). Redo steps count against maxSteps.
  • The built-in judge defaults to the agent’s own model so authentication always works; set model to use a cheaper judge.
  • Verify requires session storage (the default), it is skipped when no session substrate is available, and skipped for runs that declared themselves incomplete via report_incomplete.
  • If a redo hits an approval gate, the run suspends normally; verification resolves at the end of the resumed run.

​
Verify vs. Learning

Verify catches a bad output now; Learning makes the agent better next run. They compose: a verify critique that fixed the output this run is exactly the kind of rule worth promoting into learnings (automatic promotion is planned).