---
title: "AI Agent Stop Conditions: Decide \\\"Done\\\" with Yes/No Probabilities"
description: "Agents loop forever or declare victory too early. Split \\\"done\\\" into yes/no checks, combine probability thresholds with deterministic tests, and keep a hard step limit."
canonical: "https://system-one.dev/en/blog/agent-stop-condition"
language: "en"
---

# AI Agent Stop Conditions: Decide \"Done\" with Yes/No Probabilities

**Short answer:** combine three kinds of termination condition. **Hard limits** on steps, time and tokens stop runaway loops. **Deterministic evidence** such as test results and file state proves something was actually produced. An **independent judgment** answers "was the goal met?". Don't let the model doing the work declare itself done. Most guides stop at acceptance criteria plus hard limits; the extra layer here sits between them: split your acceptance criteria into yes/no questions, get a probability for each from a decision model, and let code combine those probabilities with the deterministic evidence to decide whether to stop, run another round or hand off to a person.

Background on Noul (yes/no) and the other primitives: [What is a decision model?](https://system-one.dev/en/blog/what-is-a-decision-model)

**Try it online (sign-in required):** [Code diff risk](https://system-one.dev/en/demo/code-review-risk)

## Three kinds of stop condition

| Type | Examples | Answers | Can't answer |
| --- | --- | --- | --- |
| Hard limit | Max steps, runtime, token or cost budget | "Should we force a stop?" | Whether the task is actually done |
| Deterministic evidence | Tests pass, exit code 0, file created, clean `git status` | "Does verifiable output exist?" | Requirements the tests don't cover |
| Semantic judgment | "Was every requested item handled?" | "Was the goal met?" | On its own, can't prove anything happened |

Existing frameworks already cover the first two:

- **Vercel AI SDK:** `ToolLoopAgent` stops after 20 steps by default (`stepCountIs(20)`), which the docs say prevents runaway loops from causing excessive API calls and cost. You can combine several `stopWhen` conditions (any one stops the loop) or write a custom one, such as stopping when cumulative token cost exceeds a budget.
- **Claude Code `/goal`:** the docs recommend `/goal` for work with a verifiable end state. A community tutorial's example is "`npm test` exits 0 and `git status` is clean, or stop after 20 turns", which puts deterministic evidence and a turn limit in one condition.

## Why the agent shouldn't declare itself done

Claude Code's design shows one approach. Per its docs, without `/goal` Claude stops when **it** judges the work complete. `/goal` adds an independent evaluator that checks your condition after every turn, so a fresh model, not the one doing the work, decides whether it's done. The evaluator reads the condition and the conversation so far, uses the configured small fast model by default (Haiku on the Claude API), and returns yes or no with one line of reasoning. On "no", Claude keeps going and uses that reason as guidance.

Same principle here: **something other than the worker judges "done"**. The difference is that we split one big condition into several yes/no questions, get a probability for each, and let code apply thresholds.

## Split "done?" into yes/no questions

Noul is a yes/no question: give a clear proposition and get P(true). The System One SDK defines it with `booleanQuestion()`, which HTTP adapters send as native `noul`; the SDK answer field is `probability`.

Writing acceptance criteria (from the System One primitives docs and the Jev Noul docs):

- **One proposition per question.** When several conditions can be true at once, define one Noul per condition rather than forcing them into a Choice. The probabilities refer to different propositions and don't need to sum to 1.
- **A low probability isn't low confidence.** 0.2 means the model thinks the condition is unlikely to hold, not that it's unsure. Noul has no separate `confidence`; values near 0.5 mean the model can't decide.
- **A probability isn't a fact.** P(true)=0.8 isn't a confirmed fact or an automatic boolean. "The output says tests passed" and "the tests passed" are different; read the test report for the second.
- **Use data you already have.** The System One docs say that if authoritative application data already settles a fact, use it rather than asking a model. Test results, exit codes and file existence belong to code.
- **Questions don't see each other.** Questions in one call evaluate the same state independently; one answer doesn't feed another. The docs note this independence is about the question interface; it doesn't guarantee that the model's errors on different questions are independent.

A starter checklist (example wording; adapt to your task):

| Acceptance criterion | Check |
| --- | --- |
| Every requirement the user listed is handled | Noul: "Does the output cover every acceptance criterion listed in the task?" |
| No blocking question left for the user | Noul: "Does the output leave no question the user must answer?" |
| All tests pass, and tests actually ran | Deterministic: read the test runner's result |
| Runtime, steps and cost within limits | Hard limit: counters in the framework or your code |

The first two rows are the two Noul questions in the example below.

## Threshold → action table

With probabilities and evidence in hand, your code decides. The Jev Noul docs give the principle: raise the threshold when a false "yes" is costly, lower it when missing a real "yes" is costly, and send the middle band to a person. Each row below maps to a branch in `05-completion-actions.ts` (order explained in the next section); the last row applies only to the Claude Code Stop hook. 0.7, 0.4, 20 steps and 3 rounds are example values; there is no universal threshold.

| Situation | Action |
| --- | --- |
| Tests fail and the model also says not done | Run another round with the failure output |
| Tests fail but the model says done | Evidence contradicts the model: hand to a person |
| No tests ran | Not done: run another round |
| Tests pass, both criteria ≥ 0.7 | Stop and report |
| Tests pass, lowest probability between 0.4 and 0.7 | Run another round |
| Tests pass, lowest probability below 0.4 | Hand to a person |
| Hit the 20-step limit, or 3 rounds with no progress | Force stop, report status, hand to a person |
| (Claude Code Stop hook only) already blocked once (`stop_hook_active` is true) and tests still fail or none ran | Stop blocking, hand to a person |

"3 rounds with no progress" is stall detection: the example builds a progress fingerprint from test results plus agent output, and three identical fingerprints in a row count as a stall. Application code implements this. Microsoft's Microsoft-Decision-1 launch blog also lists agent control as a use case: evaluating an agent's proposed next step and deciding whether to continue, stop, retry, or hand off to a model, tool or person (vendor statement).

Pick thresholds on your own human-labelled cases. For how to select on labelled samples and confirm on held-out data, see [An LLM-as-a-judge alternative](https://system-one.dev/en/blog/llm-as-a-judge-alternative).

### Implementing the action table

The dev team's `examples/decision-articles/05-completion-actions.ts` imports `checkCompletion()` and the thresholds from the main example (`05-agent-completion-check.ts`, below) and adds loop counters. `decideCompletionAction()` checks in this order and returns on the first match:

1. Step limit `MAX_AGENT_STEPS = 20` reached: hand to a person.
2. `STALL_ROUNDS = 3` identical progress fingerprints (tests + output): hand to a person.
3. No tests ran: run another round.
4. Tests failed and the lower of the two probabilities is ≥ 0.7 (model says done): evidence contradicts the model, hand to a person.
5. Tests failed and the model agrees it's not done: run another round.
6. Tests passed and the lower probability is ≥ 0.7: stop and report.
7. Tests passed and the lower probability is between `MID_BAND_LOW = 0.4` and 0.7: run another round.
8. Otherwise (lower probability under 0.4): hand to a person.

Rule 3 comes before rules 4 and 5: if the result is marked failed but zero tests ran, it counts as "no tests ran" (another round), not as a failure. All numbers are examples; tune them on your own cases.

```ts
/**
 * Article 5 — Action table: retry, human handoff, hard cap, stall detection.
 *
 * `05-agent-completion-check.ts` only returns done true/false. This file maps
 * probabilities + test evidence + loop counters onto the article action table.
 * System One still has no `done` primitive, step counter, or stall detector.
 */
import type { EvaluationClient } from '@system-one-ai/core';
import {
  MIN_CRITERION_PROBABILITY,
  type TestEvidence,
  checkCompletion,
  createHostedClient,
} from './05-agent-completion-check.ts';

/** Inclusive lower bound of the mid band. Tune on labeled cases. */
export const MID_BAND_LOW = 0.4;
export const MAX_AGENT_STEPS = 20;
export const STALL_ROUNDS = 3;

export type CompletionAction =
  | { action: 'stop_done'; reason: 'tests_and_criteria_passed' }
  | { action: 'retry'; reason: 'tests_failed' | 'no_tests_ran' | 'criteria_mid_band' }
  | {
      action: 'handoff_human';
      reason: 'criteria_low' | 'evidence_contradicts_model' | 'hard_cap' | 'stalled';
    };

export interface AgentLoopState {
  step: number;
  recentFingerprints: readonly string[];
}

export function minCriterionProbability(probabilities: {
  acceptanceCovered: number;
  noOpenQuestions: number;
}) {
  return Math.min(probabilities.acceptanceCovered, probabilities.noOpenQuestions);
}

export function fingerprintProgress(tests: TestEvidence, output: string) {
  return JSON.stringify({
    passed: tests.passed,
    failed: tests.failed,
    ran: tests.ran,
    output,
  });
}

export function isStalled(fingerprints: readonly string[], window = STALL_ROUNDS) {
  if (fingerprints.length < window) return false;
  const recent = fingerprints.slice(-window);
  return recent.every(value => value === recent[0]);
}

/**
 * Deterministic evidence wins. Mid-band retries. Low probability or a
 * contradiction (tests failed while the model is above the done threshold)
 * hands off to a person. Hard cap and stall always stop the loop.
 */
export function decideCompletionAction(
  probabilities: { acceptanceCovered: number; noOpenQuestions: number },
  tests: TestEvidence,
  loop: AgentLoopState,
): CompletionAction {
  if (loop.step >= MAX_AGENT_STEPS) {
    return { action: 'handoff_human', reason: 'hard_cap' };
  }
  if (isStalled(loop.recentFingerprints)) {
    return { action: 'handoff_human', reason: 'stalled' };
  }

  const minimum = minCriterionProbability(probabilities);
  const modelSaysDone = minimum >= MIN_CRITERION_PROBABILITY;
  const modelMid = minimum >= MID_BAND_LOW && minimum < MIN_CRITERION_PROBABILITY;

  // Flagged failed with zero runs is "no tests ran", not a real failure.
  if (tests.ran === 0) {
    return { action: 'retry', reason: 'no_tests_ran' };
  }
  if (!tests.passed) {
    if (modelSaysDone) {
      return { action: 'handoff_human', reason: 'evidence_contradicts_model' };
    }
    return { action: 'retry', reason: 'tests_failed' };
  }
  if (modelSaysDone) {
    return { action: 'stop_done', reason: 'tests_and_criteria_passed' };
  }
  if (modelMid) {
    return { action: 'retry', reason: 'criteria_mid_band' };
  }
  return { action: 'handoff_human', reason: 'criteria_low' };
}

export async function decideCompletionAfterEvaluate(
  client: EvaluationClient,
  input: {
    task: string;
    output: string;
    acceptance: readonly string[];
    tests: TestEvidence;
    loop: AgentLoopState;
  },
) {
  const checked = await checkCompletion(client, input);
  return {
    ...decideCompletionAction(checked.probabilities, input.tests, input.loop),
    probabilities: checked.probabilities,
    model: checked.model,
  };
}

async function main() {
  const tests: TestEvidence = {
    passed: process.env.EXAMPLE_TESTS_PASSED !== '0',
    failed: process.env.EXAMPLE_FAILED_TESTS ? process.env.EXAMPLE_FAILED_TESTS.split(',') : [],
    ran: Number(process.env.EXAMPLE_TESTS_RAN ?? 3),
  };
  const output = 'Implemented decideCompletionAction with mid-band retry and stall detection.';
  const decision = await decideCompletionAfterEvaluate(createHostedClient(), {
    task: 'Stop only when tests pass and both yes/no checks are above threshold.',
    output,
    acceptance: ['Failed tests prevent done', 'Mid-band retries instead of stopping'],
    tests,
    loop: {
      step: Number(process.env.EXAMPLE_STEP ?? 1),
      recentFingerprints: [fingerprintProgress(tests, output)],
    },
  });
  console.log(JSON.stringify(decision, null, 2));
}

if (process.argv[1] && import.meta.url.endsWith(process.argv[1].split(/[\\/]/).pop() ?? '')) {
  main().catch((error: unknown) => {
    console.error(error instanceof Error ? error.message : error);
    process.exitCode = 1;
  });
}
```

## Combining with AI SDK `stopWhen`

AI SDK `stopWhen` accepts custom conditions that see every executed step, and you can put them in an array with the built-in step-count condition; any one stops the loop. So keep the step limit as the hard limit and add a custom condition that reads test results and calls System One for the completion check.

`examples/decision-articles/05-ai-sdk-stop-when.ts` shows two ways: pass "step limit + async completion check" as a `stopWhen` array to `ToolLoopAgent`, or, like the AI SDK docs' manual loop, run `generateText` one step at a time and call `decideCompletionAction()` after each step. The comments note that `StopCondition` may return `PromiseLike<boolean>`, so an async System One check works.

This repo uses `ai@6.0.82`, where the step condition is `stepCountIs()`. (AI SDK 7 renames it `isStepCount()`; see the [AI SDK 7 migration guide](https://ai-sdk.dev/docs/migration-guides/migration-guide-7-0).)

```ts
/**
 * Article 5 — Combine completion checks with the Vercel AI SDK loop.
 *
 * In `ai@6` the published name is `stepCountIs` (the article draft's
 * `isStepCount` is not an export). `StopCondition` may return
 * `PromiseLike<boolean> | boolean`, so an async System One check is valid.
 * Multiple `stopWhen` entries stop when any condition is true.
 *
 * Alternative: a manual `generateText` loop (default `stopWhen` is
 * `stepCountIs(1)`) that calls `decideCompletionAction` after each step.
 */
import { ToolLoopAgent, generateText, stepCountIs } from 'ai';
import type { LanguageModel, StopCondition, ToolSet } from 'ai';
import type { EvaluationClient } from '@system-one-ai/core';
import type { TestEvidence } from './05-agent-completion-check.ts';
import {
  MAX_AGENT_STEPS,
  decideCompletionAction,
  fingerprintProgress,
  type CompletionAction,
} from './05-completion-actions.ts';
import { checkCompletion } from './05-agent-completion-check.ts';

export function hardCapStopWhen(maxSteps = MAX_AGENT_STEPS) {
  return stepCountIs(maxSteps);
}

/**
 * Custom async stop condition. Receives the AI SDK `steps` array.
 */
export function createCompletionStopWhen<TOOLS extends ToolSet>(
  inspect: (steps: Parameters<StopCondition<TOOLS>>[0]['steps']) => Promise<boolean> | boolean,
): StopCondition<TOOLS> {
  return async ({ steps }) => inspect(steps);
}

export function completionStopWhen<TOOLS extends ToolSet>(
  checkDone: (steps: Parameters<StopCondition<TOOLS>>[0]['steps']) => Promise<boolean> | boolean,
  maxSteps = MAX_AGENT_STEPS,
): Array<StopCondition<TOOLS>> {
  return [hardCapStopWhen(maxSteps), createCompletionStopWhen(checkDone)];
}

export function createCompletionAgent<TOOLS extends ToolSet>(settings: {
  model: LanguageModel;
  tools?: TOOLS;
  checkDone: (steps: Parameters<StopCondition<TOOLS>>[0]['steps']) => Promise<boolean> | boolean;
}) {
  return new ToolLoopAgent({
    model: settings.model,
    tools: settings.tools,
    stopWhen: completionStopWhen(settings.checkDone),
  });
}

export interface ManualLoopStep {
  text: string;
}

/**
 * Testable loop body. `generateStep` is your `generateText` (or any other
 * application step). System One is consulted only through `evaluateCriteria`.
 */
export async function runManualCompletionLoop(input: {
  maxSteps?: number;
  generateStep: (step: number) => Promise<ManualLoopStep>;
  readTests: () => TestEvidence;
  evaluateCriteria: (output: string) => Promise<{
    acceptanceCovered: number;
    noOpenQuestions: number;
  }>;
}) {
  const maxSteps = input.maxSteps ?? MAX_AGENT_STEPS;
  const fingerprints: string[] = [];
  let lastAction: CompletionAction = { action: 'handoff_human', reason: 'hard_cap' };
  let lastText = '';

  for (let step = 1; step <= maxSteps; step += 1) {
    const generated = await input.generateStep(step);
    lastText = generated.text;
    const tests = input.readTests();
    fingerprints.push(fingerprintProgress(tests, generated.text));
    lastAction = decideCompletionAction(
      await input.evaluateCriteria(generated.text),
      tests,
      { step, recentFingerprints: fingerprints },
    );
    if (lastAction.action !== 'retry') {
      return { ...lastAction, text: lastText, step };
    }
  }

  return { action: 'handoff_human' as const, reason: 'hard_cap' as const, text: lastText, step: maxSteps };
}

/**
 * Manual `generateText` loop from the AI SDK docs: one model step, then decide.
 */
export async function runGenerateTextCompletionLoop(input: {
  model: LanguageModel;
  prompt: string;
  tools?: ToolSet;
  client: EvaluationClient;
  task: string;
  acceptance: readonly string[];
  readTests: () => TestEvidence;
  maxSteps?: number;
}) {
  return runManualCompletionLoop({
    maxSteps: input.maxSteps,
    generateStep: async () => {
      const result = await generateText({
        model: input.model,
        prompt: input.prompt,
        tools: input.tools,
        stopWhen: stepCountIs(1),
      });
      return { text: result.text };
    },
    readTests: input.readTests,
    evaluateCriteria: async (output) => {
      const checked = await checkCompletion(input.client, {
        task: input.task,
        output,
        acceptance: input.acceptance,
        tests: input.readTests(),
      });
      return checked.probabilities;
    },
  });
}

async function main() {
  console.log(JSON.stringify({
    hardCap: 'stepCountIs(20)',
    asyncStopWhenSupported: true,
    note: 'Pass a Vercel AI SDK LanguageModel into createCompletionAgent or runGenerateTextCompletionLoop. System One is not that model.',
  }, null, 2));
}

if (process.argv[1] && import.meta.url.endsWith(process.argv[1].split(/[\\/]/).pop() ?? '')) {
  main().catch((error: unknown) => {
    console.error(error instanceof Error ? error.message : error);
    process.exitCode = 1;
  });
}
```

## TypeScript: yes/no questions + test evidence

From `examples/decision-articles/05-agent-completion-check.ts`. Two `booleanQuestion()`s ask "are all acceptance criteria covered?" and "is nothing left blocking?", combined with the test runner's result. Failed tests or zero tests mean not done; both probabilities must meet the threshold to count as done.

```ts
/**
 * Article 5 — Agent completion check: yes/no questions plus hard evidence.
 *
 * Split "is the task done?" into independent Noul questions via
 * `booleanQuestion()` (wire type `noul`, SDK answer `.probability`).
 * Combine those probabilities with deterministic evidence your runtime
 * already has — here, a test-runner summary. System One does not run
 * tests, read CI, or expose a "done" primitive.
 *
 * A low probability means the statement is estimated unlikely, not that
 * the model lacks confidence. Failed tests override a high probability.
 */
import { createSystemOne, booleanQuestion } from '@system-one-ai/core';
import type { EvaluationClient } from '@system-one-ai/core';
import { systemOneAdapter } from '@system-one-ai/adapter-system-one';
import { createFetchTransport } from '@system-one-ai/transport-fetch';

export const doneCriteria = {
  acceptanceCovered: booleanQuestion(
    'Does the agent output address every acceptance criterion listed in the task?',
    { true: 'Each written criterion has a corresponding result', false: 'At least one criterion is missing' },
  ),
  noOpenQuestions: booleanQuestion(
    'Does the agent output leave no blocking question for the user?',
    { true: 'The user can stop; nothing essential is unanswered', false: 'The user must still answer a blocking question' },
  ),
};

/** Application-owned. Tune on your own cases; this is not a product default. */
export const MIN_CRITERION_PROBABILITY = 0.7;

export interface TestEvidence {
  passed: boolean;
  failed: readonly string[];
  ran: number;
}

export function createHostedClient() {
  const apiKey = process.env.SYSTEM_ONE_API_KEY;
  const baseURL = process.env.SYSTEM_ONE_BASE_URL;
  const model = process.env.SYSTEM_ONE_MODEL;
  if (!apiKey || !baseURL || !model) {
    throw new Error('Set SYSTEM_ONE_API_KEY, SYSTEM_ONE_BASE_URL (…/v1), and SYSTEM_ONE_MODEL from GET /v1/models.');
  }
  return createSystemOne({
    apiKey, baseURL, model, adapter: systemOneAdapter,
    transport: createFetchTransport(), timeoutMs: 30_000, maxRetries: 0,
  });
}

export function combineDoneSignal(
  probabilities: { acceptanceCovered: number; noOpenQuestions: number },
  tests: TestEvidence,
) {
  const modelSaysDone =
    probabilities.acceptanceCovered >= MIN_CRITERION_PROBABILITY
    && probabilities.noOpenQuestions >= MIN_CRITERION_PROBABILITY;
  if (tests.ran === 0) {
    return { done: false, reason: 'no_tests_ran' as const, modelSaysDone };
  }
  if (!tests.passed) {
    return { done: false, reason: 'tests_failed' as const, failed: tests.failed, modelSaysDone };
  }
  if (!modelSaysDone) {
    return { done: false, reason: 'criteria_below_threshold' as const, modelSaysDone };
  }
  return { done: true, reason: 'tests_and_criteria_passed' as const, modelSaysDone };
}

export async function checkCompletion(
  client: EvaluationClient,
  input: { task: string; output: string; acceptance: readonly string[]; tests: TestEvidence },
) {
  const result = await client.evaluate({
    state: {
      task: input.task,
      output: input.output,
      acceptance: [...input.acceptance],
      testsPassed: input.tests.passed,
      testsRan: input.tests.ran,
      failedTests: [...input.tests.failed],
    },
    questions: doneCriteria,
  });
  const probabilities = {
    acceptanceCovered: result.answers.acceptanceCovered.probability,
    noOpenQuestions: result.answers.noOpenQuestions.probability,
  };
  return {
    ...combineDoneSignal(probabilities, input.tests),
    probabilities,
    model: result.model,
  };
}

async function main() {
  const tests: TestEvidence = {
    passed: process.env.EXAMPLE_TESTS_PASSED !== '0',
    failed: process.env.EXAMPLE_FAILED_TESTS ? process.env.EXAMPLE_FAILED_TESTS.split(',') : [],
    ran: Number(process.env.EXAMPLE_TESTS_RAN ?? 3),
  };
  const decision = await checkCompletion(createHostedClient(), {
    task: 'Add a refund-eligibility check that refuses when tests fail.',
    output: 'Implemented combineDoneSignal; unit tests cover failed and passing suites.',
    acceptance: [
      'Failed tests prevent a done=true result',
      'Yes/no model checks are combined with AND',
    ],
    tests,
  });
  console.log(JSON.stringify(decision, null, 2));
}

if (process.argv[1] && import.meta.url.endsWith(process.argv[1].split(/[\\/]/).pop() ?? '')) {
  main().catch((error: unknown) => {
    console.error(error instanceof Error ? error.message : error);
    process.exitCode = 1;
  });
}
```

Notes (from the code and comments):

- **System One doesn't run tests and has no `done` primitive.** It doesn't read CI. Test results are your app's deterministic evidence, passed in as `TestEvidence`.
- **Deterministic evidence wins.** However high the probabilities, `combineDoneSignal` checks tests first: `no_tests_ran` when none ran, `tests_failed` when they ran and failed.
- **The two checks are ANDed.** Both must be ≥ `MIN_CRITERION_PROBABILITY = 0.7` for `modelSaysDone`. The comment says this is "not a product default"; tune it on your cases.
- **Hosted model:** take `SYSTEM_ONE_MODEL` from `GET /v1/models`. A full model ID selects a model but can't pin a version.

For the action table (retry band, human handoff, hard limit, stall detection) see "Implementing the action table"; for AI SDK see "Combining with AI SDK `stopWhen`".

### Claude Code Stop hook

`examples/decision-articles/05-claude-code-stop-hook.ts` wraps `checkCompletion()` as a Claude Code Stop hook. It reads the hook JSON from stdin, uses `last_assistant_message` as the agent output, and runs the completion check together with the test evidence. Its decisions mostly match the table, with two differences: when tests pass but the yes/no checks fall short, it always blocks for another round (no separate "below 0.4 → person" band); and it has no step limit or stall detection. It has three outputs:

- **Block, run another round:** when tests fail and the model also says not done, when no tests ran, or when tests pass but a check is below threshold, it prints `{"decision": "block", "reason": "..."}` so Claude keeps working with that reason.
- **Hand to a person:** when tests fail but the model says done, or when `stop_hook_active` is true and tests still fail or none ran, it stops blocking, prints `{"systemMessage": "Tests failing but agent believes it is done — needs human review"}` and writes the same line to stderr, so Claude stops and a person takes over. The [hooks docs](https://code.claude.com/docs/en/hooks) define `systemMessage` as a warning shown to the user. The example prints the same message for every handoff; reword it by cause when you adopt it.
- **Allow:** when tests pass and both checks meet the threshold, it prints nothing and Claude can stop.

`stop_hook_active: true` means Claude is already continuing because of an earlier block. The example still checks tests first: failing or missing tests go to a person instead of blocking again; passing tests are allowed without re-running the yes/no checks. So it blocks at most once and then hands off, rather than relying on Claude Code's own cap. The hooks docs say that after a Stop hook has continued the turn 8 times in a row, Claude Code ignores the next block and ends the turn. That's only a backstop, and a force-ended turn has nobody picking it up, so the example doesn't rely on it. The comments say the contract was checked against the Claude Code hooks docs (2026-10-11). System One has no Claude Code integration and no `done` primitive; this is your own hook script calling the System One API.

```ts
/**
 * Article 5 — Claude Code Stop hook around System One completion checks.
 *
 * Contract verified against https://code.claude.com/docs/en/hooks (2026-10-11):
 *   Input (stdin JSON), Stop-specific: hook_event_name, stop_hook_active,
 *   last_assistant_message, plus common session_id / transcript_path / cwd /
 *   permission_mode. Use last_assistant_message; do not parse transcript_path.
 *   `stop_hook_active` is true when Claude is already continuing because a
 *   previous Stop hook blocked. This file still reads test results: failed
 *   tests hand off to a person (allow stop + `systemMessage` / stderr) so the
 *   hook cannot loop; passing tests allow stop. Claude Code also applies an
 *   8-consecutive-continuation cap.
 *   Output: omit JSON to allow stop; or
 *     { "decision": "block", "reason": "..." }
 *   or, when handing off without blocking:
 *     { "systemMessage": "..." }
 *   `reason` is required when decision is "block". Exit 0 with that JSON.
 *   Exit 2 + stderr also blocks; this file uses the JSON form.
 *   hookSpecificOutput.additionalContext is the non-error continue path —
 *   this hook uses decision/reason when Claude should keep working.
 *
 * System One still has no Claude Code integration or `done` primitive. Test
 * results are application evidence. Merge 05-claude-code-stop-hook.settings.json
 * into project `.claude/settings.json`.
 */
import type { EvaluationClient } from '@system-one-ai/core';
import {
  type TestEvidence,
  checkCompletion,
  createHostedClient,
} from './05-agent-completion-check.ts';

/** Documented Stop input fields used by this hook. Extra keys are ignored. */
export interface StopHookInput {
  hook_event_name?: string;
  session_id?: string;
  transcript_path?: string;
  cwd?: string;
  permission_mode?: string;
  stop_hook_active?: boolean;
  last_assistant_message?: string;
}

/** Documented Stop block output. The only `decision` value is `"block"`. */
export interface StopHookBlockOutput {
  decision: 'block';
  reason: string;
}

/** Allow stop and show a user-visible note. No `decision`, so Claude may stop. */
export interface StopHookHumanHandoff {
  systemMessage: string;
}

export type StopHookResult = StopHookBlockOutput | StopHookHumanHandoff | undefined;

/** Exact copy for the tests-failed / model-says-done handoff. */
export const HUMAN_REVIEW_NOTE = 'Tests failing but agent believes it is done — needs human review';

export function isStopHookHumanHandoff(result: StopHookResult): result is StopHookHumanHandoff {
  return Boolean(result && 'systemMessage' in result && !('decision' in result));
}

export function parseStopHookInput(raw: unknown): StopHookInput {
  if (typeof raw !== 'object' || raw === null || Array.isArray(raw)) {
    throw new Error('Stop hook stdin must be a JSON object.');
  }
  const record = raw as Record<string, unknown>;
  const text = (key: string) => {
    const value = record[key];
    return typeof value === 'string' ? value : undefined;
  };
  return {
    hook_event_name: text('hook_event_name'),
    session_id: text('session_id'),
    transcript_path: text('transcript_path'),
    cwd: text('cwd'),
    permission_mode: text('permission_mode'),
    stop_hook_active: typeof record.stop_hook_active === 'boolean' ? record.stop_hook_active : undefined,
    last_assistant_message: text('last_assistant_message'),
  };
}

export function readTestEvidenceFromEnv(env: NodeJS.ProcessEnv = process.env): TestEvidence {
  return {
    passed: env.EXAMPLE_TESTS_PASSED !== '0',
    failed: env.EXAMPLE_FAILED_TESTS ? env.EXAMPLE_FAILED_TESTS.split(',').filter(Boolean) : [],
    ran: Number(env.EXAMPLE_TESTS_RAN ?? 0),
  };
}

export function blockReasonFromCompletion(result: {
  done: boolean;
  reason: string;
  failed?: readonly string[];
  probabilities?: { acceptanceCovered: number; noOpenQuestions: number };
}) {
  if (result.reason === 'tests_failed') {
    const failed = result.failed?.length ? result.failed.join(', ') : 'unspecified tests';
    return `Stop blocked: tests failed (${failed}). Fix the failures before finishing.`;
  }
  if (result.reason === 'no_tests_ran') {
    return 'Stop blocked: no tests ran. Run the test suite and try again.';
  }
  if (result.reason === 'criteria_below_threshold') {
    const covered = result.probabilities?.acceptanceCovered;
    const open = result.probabilities?.noOpenQuestions;
    return `Stop blocked: yes/no completion checks are below threshold (acceptanceCovered=${covered}, noOpenQuestions=${open}). Address remaining criteria or questions.`;
  }
  return `Stop blocked: ${result.reason}.`;
}

/**
 * Map tests + model signal onto the documented Stop decision.
 *
 * (a) Tests failed and the model does not say done → block, another round.
 * (b) Tests failed but the model says done → allow stop + human-review note.
 * (c) No tests ran → block, another round.
 * When `stop_hook_active` is true, failed tests or zero runs hand off (b)
 * so the hook cannot loop; passing tests allow stop.
 */
export function decideStopHook(
  input: StopHookInput,
  checked: {
    done: boolean;
    reason: string;
    modelSaysDone: boolean;
    failed?: readonly string[];
    probabilities?: { acceptanceCovered: number; noOpenQuestions: number };
  },
  tests: TestEvidence,
): StopHookResult {
  if (input.hook_event_name !== undefined && input.hook_event_name !== 'Stop') {
    return undefined;
  }

  const noTestsRan = tests.ran === 0;
  const testsFailed = tests.ran > 0 && !tests.passed;

  if (testsFailed) {
    if (input.stop_hook_active === true || checked.modelSaysDone) {
      return { systemMessage: HUMAN_REVIEW_NOTE };
    }
    return { decision: 'block', reason: blockReasonFromCompletion({ ...checked, reason: 'tests_failed' }) };
  }

  if (noTestsRan) {
    if (input.stop_hook_active === true) {
      return { systemMessage: HUMAN_REVIEW_NOTE };
    }
    return { decision: 'block', reason: blockReasonFromCompletion({ ...checked, reason: 'no_tests_ran' }) };
  }

  if (input.stop_hook_active === true || checked.done) {
    return undefined;
  }
  return { decision: 'block', reason: blockReasonFromCompletion(checked) };
}

export function stopHookOutputFromCompletion(result: {
  done: boolean;
  reason: string;
  modelSaysDone?: boolean;
  failed?: readonly string[];
  probabilities?: { acceptanceCovered: number; noOpenQuestions: number };
}): StopHookResult {
  return decideStopHook(
    { hook_event_name: 'Stop', stop_hook_active: false },
    { ...result, modelSaysDone: result.modelSaysDone ?? false },
    {
      passed: result.reason !== 'tests_failed' && result.reason !== 'no_tests_ran',
      failed: result.failed ?? [],
      ran: result.reason === 'no_tests_ran' ? 0 : 1,
    },
  );
}

export async function evaluateStopHook(
  input: StopHookInput,
  client: EvaluationClient,
  context: {
    task: string;
    acceptance: readonly string[];
    tests: TestEvidence;
  },
): Promise<StopHookResult> {
  if (input.hook_event_name !== undefined && input.hook_event_name !== 'Stop') {
    return undefined;
  }
  const checked = await checkCompletion(client, {
    task: context.task,
    output: input.last_assistant_message ?? '',
    acceptance: context.acceptance,
    tests: context.tests,
  });
  return decideStopHook(input, checked, context.tests);
}

async function readStdin() {
  const chunks: Buffer[] = [];
  for await (const chunk of process.stdin) chunks.push(Buffer.from(chunk));
  const text = Buffer.concat(chunks).toString('utf8').trim();
  return text === '' ? {} : JSON.parse(text) as unknown;
}

async function main() {
  const input = parseStopHookInput(await readStdin());
  const output = await evaluateStopHook(input, createHostedClient(), {
    task: process.env.EXAMPLE_TASK ?? 'Complete the requested work and leave no blocking questions.',
    acceptance: process.env.EXAMPLE_ACCEPTANCE
      ? process.env.EXAMPLE_ACCEPTANCE.split('|').filter(Boolean)
      : ['Acceptance criteria are covered', 'No blocking questions remain'],
    tests: readTestEvidenceFromEnv(),
  });
  if (isStopHookHumanHandoff(output)) {
    console.error(output.systemMessage);
    process.stdout.write(`${JSON.stringify(output)}\n`);
    return;
  }
  if (output) process.stdout.write(`${JSON.stringify(output)}\n`);
}

if (process.argv[1] && import.meta.url.endsWith(process.argv[1].split(/[\\/]/).pop() ?? '')) {
  main().catch((error: unknown) => {
    console.error(error instanceof Error ? error.message : error);
    process.exitCode = 1;
  });
}
```

Then merge this config (`examples/decision-articles/05-claude-code-stop-hook.settings.json`) into your project's `.claude/settings.json`:

```json
{
  "hooks": {
    "Stop": [
      {
        "hooks": [
          {
            "type": "command",
            "command": "node",
            "args": [
              "--experimental-strip-types",
              "${CLAUDE_PROJECT_DIR}/examples/decision-articles/05-claude-code-stop-hook.ts"
            ]
          }
        ]
      }
    ]
  }
}
```

The hook and `/goal` are different paths: `/goal` uses Claude Code's built-in evaluator; the Stop hook lets your own test evidence and System One yes/no probabilities decide when to stop.

> These stack: hard limits as the backstop, deterministic checks first, semantic judgment for what tests don't cover. We haven't compared these approaches head to head, so we give no cost or accuracy figures.

Hosted billing: every 4KB of request is one block, and anything under 4KB still counts as one; cost = blocks × questions × the model's price per block. Two yes/no questions per check means two questions' worth of blocks. New accounts get 100 free credits on sign-up; see [pricing](https://system-one.dev/en/pricing).

## Related reading
- [What is a decision model?](https://system-one.dev/en/blog/what-is-a-decision-model)
- [Jev alternatives in 2026](https://system-one.dev/en/blog/jev-alternatives)
- [An LLM-as-a-judge alternative](https://system-one.dev/en/blog/llm-as-a-judge-alternative)
- [Route agent tool calls with a Choice question](https://system-one.dev/en/blog/agent-tool-routing)
- [System One primitives docs](https://docs.system-one.dev/en/docs/primitives)


## Who does what (checked 2026-10-11)

| Mechanism | How it decides | Limits | Source |
| --- | --- | --- | --- |
| AI SDK `stopWhen` hard limit<br><small>[AI SDK 6 loop control](https://ai-sdk.dev/v6/docs/agents/loop-control) · 2026-10-11</small> | `ToolLoopAgent` default `stepCountIs(20)`; combine conditions, any one stops | Bounds the loop; doesn't judge completion | Official docs |
| AI SDK `done` tool<br><small>[AI SDK 6 loop control](https://ai-sdk.dev/v6/docs/agents/loop-control) · 2026-10-11</small> | `toolChoice: 'required'` plus a `done` tool without `execute`; the loop stops when it's called | The working model declares itself done | Official docs |
| Claude Code `/goal`<br><small>[Claude Code docs](https://code.claude.com/docs/en/goal) · 2026-10-11</small> | After each turn, an independent small fast model returns yes/no plus a reason | Claude Code sessions only; judges from the conversation | Official docs |
| Deterministic checks (tests, exit codes, files)<br><small>[System One primitives](https://docs.system-one.dev/en/docs/primitives) · 2026-10-11</small> | Code reads results directly; authoritative data first | Only proves what's checked | Official docs + our recommendation |
| Decision model Noul (several yes/no questions)<br><small>[Jev Noul docs](https://docs.typesafe.ai/primitives/noul.md) · 2026-10-11</small> | P(true) per proposition; app sets thresholds by risk; middle band to a person | A probability isn't a confirmed fact and doesn't authorize actions<br><small>[System One primitives](https://docs.system-one.dev/en/docs/primitives) · 2026-10-11</small> | Official docs |
| System One hosted models<br><small>[System One docs](https://docs.system-one.dev/en/docs) · 2026-10-11</small> | SDK `booleanQuestion()` returns `probability` | system-one.dev offers Jev, Clef Flash and Clef; exact IDs from `GET /v1/models`<br><small>[System One API](https://system-one.dev/en/api) · 2026-10-11</small> | Official docs |
## Frequently asked questions

### What kinds of agent termination conditions are there?

Three: hard limits (steps, time, tokens, cost), deterministic evidence (tests, exit codes, file state) and semantic judgment (was the goal met?). Hard limits stop runaways, evidence confirms output, and judgment covers requirements tests miss.

### Why not let the working model say it's done?

It tends to declare victory too early or loop on failing tests. Claude Code's `/goal` uses an independent evaluator after each turn, so a fresh model, not the worker, decides.

### How do I write acceptance criteria as Noul questions?

One clear proposition per question, such as "does the output cover every acceptance criterion?" or "is no question left for the user?". Make a separate question per condition, each with its own P(true). Facts with deterministic answers (test results, file existence) are checked by code, not inferred by the model.

### What action goes with each probability?

In the example: tests pass and both probabilities ≥ 0.7 → stop and report; lowest between 0.4 and 0.7 → another round; lower → a person. Tests fail: another round if the model agrees it's not done, a person if the model says done. No tests ran: another round, even if the result is marked failed. Step limit or stall: a person. Set thresholds by error cost on your own cases; there's no universal value.

### How do yes/no probabilities combine with hard limits and deterministic checks?

Deterministic checks win: failed or missing tests mean not done regardless of probability. When tests fail but the model says done, both examples hand to a person. The Stop hook example blocks at most once and then hands off; Claude Code's 8-in-a-row cap is only a backstop. The hard limit always stays. Probabilities only judge semantic requirements tests can't cover.

### Does this work with AI SDK `stopWhen`?

Yes. Keep the step limit and add an async custom condition for the completion check, or use the AI SDK manual loop and check after each step. The examples show both, on `ai@6` with `stepCountIs()`.