Agent 什么时候算完成？用是非概率设计停止条件（TypeScript）

Noul（是非题）和另外两种原语的基础知识见 决策模型是什么？。

Agent 停在哪：三类停止条件

| 类型 | 例子 | 能回答什么 | 回答不了什么 | | --- | --- | --- | --- | | 硬上限 | 最大步数、运行时间、token 或费用预算 | “该不该强制停下” | 任务是否真的完成 | | 确定性证据 | 测试通过、退出码为 0、文件已生成、git status 干净 | “可验证的产出是否存在” | 测试没覆盖到的要求 | | 语义判断 | “用户要求的每一项都处理了吗？” | “目标是否达成” | 单独使用时不能证明事情确实发生了 |

现成的框架已经覆盖了前两类：

• Vercel AI SDK：ToolLoopAgent 默认在 20 步后停止（stepCountIs(20)），文档称这个默认值是为了防止失控循环带来过多的 API 调用和费用。你可以组合多个 stopWhen 条件，满足任意一个就停；也可以写自定义条件，比如累计 token 费用超过预算时停止。
• Claude Code /goal：官方文档建议，/goal 适合有“可验证终态”的工作。社区教程给出的例子是“npm test 退出码为 0 且 git status 干净，或 20 轮后停止”，把确定性证据和轮数上限写在同一个条件里。

为什么不能让 Agent 自己宣布完成

开发者社区的抱怨很集中。有人写道：“YOUR AI AGENT DOESN'T NEED MORE TOOLS. It needs a way to KNOW when it's DONE.”意思是 Agent 不缺工具，缺的是知道自己什么时候做完了（原帖）。也有人问：“How do you decide an agent is done rather than looping on flaky tests?”也就是怎么判断 Agent 做完了，而不是在不稳定的测试上反复循环（原帖）。

Claude Code 的设计给出了一种思路。按官方文档的说法：没有 /goal 时，Claude 在自己判断工作完成时停止；/goal 增加了一个独立的评估器，每轮结束后检查你写的条件，“由一个全新的模型而不是正在干活的模型来决定是否完成”。评估器读取完成条件和到目前为止的对话，默认使用配置的小型快速模型（Claude API 上默认是 Haiku），给出“是”或“否”加一句理由。“否”时 Claude 继续工作，并把理由当作下一轮的指导。

这和本文的思路一致：让执行者之外的东西来判断“完成了没有”。区别在于，本文把一个大的完成条件拆成多个是非问题，每个问题都拿到一个概率，再交给代码按阈值处理。

把“完成了吗”拆成几个是非问题

决策模型的 Noul 原语就是是非题：给出一个明确的命题，返回它为真的概率 P(true)。System One SDK 用 booleanQuestion() 定义这类问题，通过 HTTP 适配器发送时会转换成原生的 noul，SDK 返回的答案字段是 probability。

写完成标准时有几点要注意（均出自 System One 决策原语文档和 TypeSafe Noul 文档）：

• 一个问题只问一件事。 多个条件可以同时成立时，为每个条件分别定义一个 Noul 问题，不要硬塞进一个 Choice。这些概率对应不同的命题，不要求相加等于 1。
• 概率低不等于“没信心”。 0.2 的意思是模型估计这个条件不太可能成立，不代表模型对答案缺乏信心。Noul 没有单独的 confidence 字段，接近 0.5 才表示模型在“是”和“否”之间拿不准。
• 概率不是事实。 P(true)=0.8 不是已确认的事实，也不是自动的布尔结果。“输出声称测试通过”和“测试真的通过了”是两回事，后者要读测试报告。
• 已经有确定答案的，直接用数据。 System One 文档写道：如果权威的应用数据已经明确给出某个事实，应该直接使用，不必再让模型推断。所以测试结果、退出码、文件是否存在，都应该由代码直接检查，而不是交给模型去猜。
• 问题之间互不参考。 同一次调用里的问题各自独立评估同一份上下文，一个答案不会成为另一个问题的依据。System One 文档也提醒，这里的“独立”说的是问题接口，并不保证模型在不同问题上的错误相互独立。

一个可以照着改的完成标准清单（问题措辞仅为示例，需要按你的任务调整）：

| 完成标准 | 用什么检查 | | --- | --- | | 用户列出的每一项要求都已处理 | Noul：“输出是否覆盖了任务中列出的每一条验收标准？” | | 没有留下需要用户回答的阻塞问题 | Noul：“输出是否没有留下必须由用户回答的问题？” | | 测试全部通过，且确实跑过测试 | 确定性检查：读取测试运行器的结果 | | 运行时间、步数、费用没有超限 | 硬上限：框架或代码里的计数器 |

前两行正是下文开发团队示例中的两个 Noul 问题。

阈值 → 动作表

拿到概率和证据之后，由应用代码决定下一步。TypeSafe 的 Noul 文档给出了设阈值的原则：误判为“是”的代价高时，阈值往上调；漏掉真正的“是”代价高时，阈值往下调；落在中间的结果可以交给人工。下表各行与下文示例 05-completion-actions.ts 的分支一一对应（判断顺序见『动作表的写法』）；最后一行只适用于 Claude Code Stop hook 示例。表中的 0.7、0.4、20 步和 3 轮都是示例值，阈值没有通用值：

| 情况 | 动作 | | --- | --- | | 测试失败，模型也认为没完成 | 再跑一轮，把失败信息交给 Agent | | 测试失败，但模型认为已完成 | 证据与模型矛盾，交给人工 | | 一个测试都没跑 | 不算完成，再跑一轮 | | 测试通过，两个完成标准的概率都不低于 0.7 | 停止并汇报 | | 测试通过，最低的概率在 0.4 到 0.7 之间 | 再跑一轮 | | 测试通过，最低的概率低于 0.4 | 交给人工 | | 达到 20 步上限，或连续 3 轮没有进展 | 强制停止，汇报现状并交给人工 | | （仅 Claude Code Stop hook）已经被阻止过一次（stop_hook_active 为 true），测试仍失败或一个测试都没跑 | 不再阻止，交给人工 |

“连续 3 轮没有进展”是一个停滞检测：示例把测试结果和 Agent 输出拼成进展指纹，连续 3 轮完全相同就视为停滞，这条判断由应用代码实现。Microsoft 在 Microsoft-Decision-1 的发布博客中也把“Agent 控制”列为用途之一：评估 Agent 提议的下一步，决定继续、停止、重试，还是交给模型、工具或人工（厂商说法）。

阈值要在你自己带人工标注的案例上选。具体怎么用标注样本选阈值、再用留出数据复核，参见 用 Score 替代 LLM 当裁判中的方法。

动作表的写法

开发团队提供了一个实现上表的示例（examples/decision-articles/05-completion-actions.ts）。它从下文“TypeScript 示例”一节的主示例 05-agent-completion-check.ts 导入 checkCompletion() 和阈值，再加上循环计数。decideCompletionAction() 按下面的顺序判断，命中一条就返回：

1. 达到步数上限 MAX_AGENT_STEPS = 20：交给人工。
2. 连续 STALL_ROUNDS = 3 轮进展指纹（测试结果 + 输出）完全相同：交给人工。
3. 一个测试都没跑：再跑一轮。
4. 测试失败，且两个概率中较低的一个不低于 0.7（模型认为已完成）：视为证据与模型矛盾，交给人工。
5. 测试失败，模型也认为没完成：再跑一轮。
6. 测试通过，较低的概率不低于 0.7：停止并汇报。
7. 测试通过，较低的概率在 MID_BAND_LOW = 0.4 到 0.7 之间：再跑一轮。
8. 其余情况（较低的概率低于 0.4）：交给人工。

注意第 3 条排在第 4、5 条之前：测试结果标记为失败、但一个测试都没跑时，按“没跑测试”处理，再跑一轮，不算测试失败。这些数字都是示例值，要在自己的案例上调。

ts/**
 * Article 5 — Action table: retry, human handoff, hard cap, stall detection.
 *
 * `05-agent-completion-check.ts` only returns done true/false. This file maps
 * probabilities + test evidence + loop counters onto the article action table.
 * System One still has no `done` primitive, step counter, or stall detector.
 */
import type { EvaluationClient } from '@system-one-ai/core';
import {
  MIN_CRITERION_PROBABILITY,
  type TestEvidence,
  checkCompletion,
  createHostedClient,
} from './05-agent-completion-check.ts';

/** Inclusive lower bound of the mid band. Tune on labeled cases. */
export const MID_BAND_LOW = 0.4;
export const MAX_AGENT_STEPS = 20;
export const STALL_ROUNDS = 3;

export type CompletionAction =
  | { action: 'stop_done'; reason: 'tests_and_criteria_passed' }
  | { action: 'retry'; reason: 'tests_failed' | 'no_tests_ran' | 'criteria_mid_band' }
  | {
      action: 'handoff_human';
      reason: 'criteria_low' | 'evidence_contradicts_model' | 'hard_cap' | 'stalled';
    };

export interface AgentLoopState {
  step: number;
  recentFingerprints: readonly string[];
}

export function minCriterionProbability(probabilities: {
  acceptanceCovered: number;
  noOpenQuestions: number;
}) {
  return Math.min(probabilities.acceptanceCovered, probabilities.noOpenQuestions);
}

export function fingerprintProgress(tests: TestEvidence, output: string) {
  return JSON.stringify({
    passed: tests.passed,
    failed: tests.failed,
    ran: tests.ran,
    output,
  });
}

export function isStalled(fingerprints: readonly string[], window = STALL_ROUNDS) {
  if (fingerprints.length < window) return false;
  const recent = fingerprints.slice(-window);
  return recent.every(value => value === recent[0]);
}

/**
 * Deterministic evidence wins. Mid-band retries. Low probability or a
 * contradiction (tests failed while the model is above the done threshold)
 * hands off to a person. Hard cap and stall always stop the loop.
 */
export function decideCompletionAction(
  probabilities: { acceptanceCovered: number; noOpenQuestions: number },
  tests: TestEvidence,
  loop: AgentLoopState,
): CompletionAction {
  if (loop.step >= MAX_AGENT_STEPS) {
    return { action: 'handoff_human', reason: 'hard_cap' };
  }
  if (isStalled(loop.recentFingerprints)) {
    return { action: 'handoff_human', reason: 'stalled' };
  }

  const minimum = minCriterionProbability(probabilities);
  const modelSaysDone = minimum >= MIN_CRITERION_PROBABILITY;
  const modelMid = minimum >= MID_BAND_LOW && minimum < MIN_CRITERION_PROBABILITY;

  // Flagged failed with zero runs is "no tests ran", not a real failure.
  if (tests.ran === 0) {
    return { action: 'retry', reason: 'no_tests_ran' };
  }
  if (!tests.passed) {
    if (modelSaysDone) {
      return { action: 'handoff_human', reason: 'evidence_contradicts_model' };
    }
    return { action: 'retry', reason: 'tests_failed' };
  }
  if (modelSaysDone) {
    return { action: 'stop_done', reason: 'tests_and_criteria_passed' };
  }
  if (modelMid) {
    return { action: 'retry', reason: 'criteria_mid_band' };
  }
  return { action: 'handoff_human', reason: 'criteria_low' };
}

export async function decideCompletionAfterEvaluate(
  client: EvaluationClient,
  input: {
    task: string;
    output: string;
    acceptance: readonly string[];
    tests: TestEvidence;
    loop: AgentLoopState;
  },
) {
  const checked = await checkCompletion(client, input);
  return {
    ...decideCompletionAction(checked.probabilities, input.tests, input.loop),
    probabilities: checked.probabilities,
    model: checked.model,
  };
}

async function main() {
  const tests: TestEvidence = {
    passed: process.env.EXAMPLE_TESTS_PASSED !== '0',
    failed: process.env.EXAMPLE_FAILED_TESTS ? process.env.EXAMPLE_FAILED_TESTS.split(',') : [],
    ran: Number(process.env.EXAMPLE_TESTS_RAN ?? 3),
  };
  const output = 'Implemented decideCompletionAction with mid-band retry and stall detection.';
  const decision = await decideCompletionAfterEvaluate(createHostedClient(), {
    task: 'Stop only when tests pass and both yes/no checks are above threshold.',
    output,
    acceptance: ['Failed tests prevent done', 'Mid-band retries instead of stopping'],
    tests,
    loop: {
      step: Number(process.env.EXAMPLE_STEP ?? 1),
      recentFingerprints: [fingerprintProgress(tests, output)],
    },
  });
  console.log(JSON.stringify(decision, null, 2));
}

if (process.argv[1] && import.meta.url.endsWith(process.argv[1].split(/[\\/]/).pop() ?? '')) {
  main().catch((error: unknown) => {
    console.error(error instanceof Error ? error.message : error);
    process.exitCode = 1;
  });
}

和 AI SDK stopWhen 怎么组合

AI SDK 的 stopWhen 支持自定义停止条件：自定义条件能拿到所有已执行步骤的信息，可以和内置的步数条件放在一个数组里，满足任意一个就停止。所以可以保留步数上限作为硬上限，再加一个自定义条件，在里面读取测试结果并调用 System One 做完成判断。

开发团队的示例（examples/decision-articles/05-ai-sdk-stop-when.ts）给出了两种写法：一是把“步数上限 + 异步完成检查”作为 stopWhen 数组交给 ToolLoopAgent；二是像 AI SDK 文档里的“手动循环”那样，用 generateText 每次只跑一步，每步之后调用上文的 decideCompletionAction()。示例注释写明，StopCondition 可以返回 PromiseLike<boolean>，所以异步的 System One 检查是可以用的。

本仓库使用 ai@6.0.82，步数条件写作 stepCountIs()。（AI SDK 7 把它改名为 isStepCount()，见 AI SDK 7 迁移指南。）

ts/**
 * Article 5 — Combine completion checks with the Vercel AI SDK loop.
 *
 * In `ai@6` the published name is `stepCountIs` (the article draft's
 * `isStepCount` is not an export). `StopCondition` may return
 * `PromiseLike<boolean> | boolean`, so an async System One check is valid.
 * Multiple `stopWhen` entries stop when any condition is true.
 *
 * Alternative: a manual `generateText` loop (default `stopWhen` is
 * `stepCountIs(1)`) that calls `decideCompletionAction` after each step.
 */
import { ToolLoopAgent, generateText, stepCountIs } from 'ai';
import type { LanguageModel, StopCondition, ToolSet } from 'ai';
import type { EvaluationClient } from '@system-one-ai/core';
import type { TestEvidence } from './05-agent-completion-check.ts';
import {
  MAX_AGENT_STEPS,
  decideCompletionAction,
  fingerprintProgress,
  type CompletionAction,
} from './05-completion-actions.ts';
import { checkCompletion } from './05-agent-completion-check.ts';

export function hardCapStopWhen(maxSteps = MAX_AGENT_STEPS) {
  return stepCountIs(maxSteps);
}

/**
 * Custom async stop condition. Receives the AI SDK `steps` array.
 */
export function createCompletionStopWhen<TOOLS extends ToolSet>(
  inspect: (steps: Parameters<StopCondition<TOOLS>>[0]['steps']) => Promise<boolean> | boolean,
): StopCondition<TOOLS> {
  return async ({ steps }) => inspect(steps);
}

export function completionStopWhen<TOOLS extends ToolSet>(
  checkDone: (steps: Parameters<StopCondition<TOOLS>>[0]['steps']) => Promise<boolean> | boolean,
  maxSteps = MAX_AGENT_STEPS,
): Array<StopCondition<TOOLS>> {
  return [hardCapStopWhen(maxSteps), createCompletionStopWhen(checkDone)];
}

export function createCompletionAgent<TOOLS extends ToolSet>(settings: {
  model: LanguageModel;
  tools?: TOOLS;
  checkDone: (steps: Parameters<StopCondition<TOOLS>>[0]['steps']) => Promise<boolean> | boolean;
}) {
  return new ToolLoopAgent({
    model: settings.model,
    tools: settings.tools,
    stopWhen: completionStopWhen(settings.checkDone),
  });
}

export interface ManualLoopStep {
  text: string;
}

/**
 * Testable loop body. `generateStep` is your `generateText` (or any other
 * application step). System One is consulted only through `evaluateCriteria`.
 */
export async function runManualCompletionLoop(input: {
  maxSteps?: number;
  generateStep: (step: number) => Promise<ManualLoopStep>;
  readTests: () => TestEvidence;
  evaluateCriteria: (output: string) => Promise<{
    acceptanceCovered: number;
    noOpenQuestions: number;
  }>;
}) {
  const maxSteps = input.maxSteps ?? MAX_AGENT_STEPS;
  const fingerprints: string[] = [];
  let lastAction: CompletionAction = { action: 'handoff_human', reason: 'hard_cap' };
  let lastText = '';

  for (let step = 1; step <= maxSteps; step += 1) {
    const generated = await input.generateStep(step);
    lastText = generated.text;
    const tests = input.readTests();
    fingerprints.push(fingerprintProgress(tests, generated.text));
    lastAction = decideCompletionAction(
      await input.evaluateCriteria(generated.text),
      tests,
      { step, recentFingerprints: fingerprints },
    );
    if (lastAction.action !== 'retry') {
      return { ...lastAction, text: lastText, step };
    }
  }

  return { action: 'handoff_human' as const, reason: 'hard_cap' as const, text: lastText, step: maxSteps };
}

/**
 * Manual `generateText` loop from the AI SDK docs: one model step, then decide.
 */
export async function runGenerateTextCompletionLoop(input: {
  model: LanguageModel;
  prompt: string;
  tools?: ToolSet;
  client: EvaluationClient;
  task: string;
  acceptance: readonly string[];
  readTests: () => TestEvidence;
  maxSteps?: number;
}) {
  return runManualCompletionLoop({
    maxSteps: input.maxSteps,
    generateStep: async () => {
      const result = await generateText({
        model: input.model,
        prompt: input.prompt,
        tools: input.tools,
        stopWhen: stepCountIs(1),
      });
      return { text: result.text };
    },
    readTests: input.readTests,
    evaluateCriteria: async (output) => {
      const checked = await checkCompletion(input.client, {
        task: input.task,
        output,
        acceptance: input.acceptance,
        tests: input.readTests(),
      });
      return checked.probabilities;
    },
  });
}

async function main() {
  console.log(JSON.stringify({
    hardCap: 'stepCountIs(20)',
    asyncStopWhenSupported: true,
    note: 'Pass a Vercel AI SDK LanguageModel into createCompletionAgent or runGenerateTextCompletionLoop. System One is not that model.',
  }, null, 2));
}

if (process.argv[1] && import.meta.url.endsWith(process.argv[1].split(/[\\/]/).pop() ?? '')) {
  main().catch((error: unknown) => {
    console.error(error instanceof Error ? error.message : error);
    process.exitCode = 1;
  });
}

TypeScript 示例：是非问题 + 测试证据

在线试一下：代码 diff 风险（示例结果，登录后可跑真实数据）

下面这段代码来自开发团队提供的示例（examples/decision-articles/05-agent-completion-check.ts）。它用两个 booleanQuestion() 判断“验收标准是否都已覆盖”和“是否没有阻塞问题”，再和测试运行器的结果组合：测试失败或一个测试都没跑时，直接判定为未完成；两个概率都达到阈值，才算完成。

ts/**
 * Article 5 — Agent completion check: yes/no questions plus hard evidence.
 *
 * Split "is the task done?" into independent Noul questions via
 * `booleanQuestion()` (wire type `noul`, SDK answer `.probability`).
 * Combine those probabilities with deterministic evidence your runtime
 * already has — here, a test-runner summary. System One does not run
 * tests, read CI, or expose a "done" primitive.
 *
 * A low probability means the statement is estimated unlikely, not that
 * the model lacks confidence. Failed tests override a high probability.
 */
import { createSystemOne, booleanQuestion } from '@system-one-ai/core';
import type { EvaluationClient } from '@system-one-ai/core';
import { systemOneAdapter } from '@system-one-ai/adapter-system-one';
import { createFetchTransport } from '@system-one-ai/transport-fetch';

export const doneCriteria = {
  acceptanceCovered: booleanQuestion(
    'Does the agent output address every acceptance criterion listed in the task?',
    { true: 'Each written criterion has a corresponding result', false: 'At least one criterion is missing' },
  ),
  noOpenQuestions: booleanQuestion(
    'Does the agent output leave no blocking question for the user?',
    { true: 'The user can stop; nothing essential is unanswered', false: 'The user must still answer a blocking question' },
  ),
};

/** Application-owned. Tune on your own cases; this is not a product default. */
export const MIN_CRITERION_PROBABILITY = 0.7;

export interface TestEvidence {
  passed: boolean;
  failed: readonly string[];
  ran: number;
}

export function createHostedClient() {
  const apiKey = process.env.SYSTEM_ONE_API_KEY;
  const baseURL = process.env.SYSTEM_ONE_BASE_URL;
  const model = process.env.SYSTEM_ONE_MODEL;
  if (!apiKey || !baseURL || !model) {
    throw new Error('Set SYSTEM_ONE_API_KEY, SYSTEM_ONE_BASE_URL (…/v1), and SYSTEM_ONE_MODEL from GET /v1/models.');
  }
  return createSystemOne({
    apiKey, baseURL, model, adapter: systemOneAdapter,
    transport: createFetchTransport(), timeoutMs: 30_000, maxRetries: 0,
  });
}

export function combineDoneSignal(
  probabilities: { acceptanceCovered: number; noOpenQuestions: number },
  tests: TestEvidence,
) {
  const modelSaysDone =
    probabilities.acceptanceCovered >= MIN_CRITERION_PROBABILITY
    && probabilities.noOpenQuestions >= MIN_CRITERION_PROBABILITY;
  if (tests.ran === 0) {
    return { done: false, reason: 'no_tests_ran' as const, modelSaysDone };
  }
  if (!tests.passed) {
    return { done: false, reason: 'tests_failed' as const, failed: tests.failed, modelSaysDone };
  }
  if (!modelSaysDone) {
    return { done: false, reason: 'criteria_below_threshold' as const, modelSaysDone };
  }
  return { done: true, reason: 'tests_and_criteria_passed' as const, modelSaysDone };
}

export async function checkCompletion(
  client: EvaluationClient,
  input: { task: string; output: string; acceptance: readonly string[]; tests: TestEvidence },
) {
  const result = await client.evaluate({
    state: {
      task: input.task,
      output: input.output,
      acceptance: [...input.acceptance],
      testsPassed: input.tests.passed,
      testsRan: input.tests.ran,
      failedTests: [...input.tests.failed],
    },
    questions: doneCriteria,
  });
  const probabilities = {
    acceptanceCovered: result.answers.acceptanceCovered.probability,
    noOpenQuestions: result.answers.noOpenQuestions.probability,
  };
  return {
    ...combineDoneSignal(probabilities, input.tests),
    probabilities,
    model: result.model,
  };
}

async function main() {
  const tests: TestEvidence = {
    passed: process.env.EXAMPLE_TESTS_PASSED !== '0',
    failed: process.env.EXAMPLE_FAILED_TESTS ? process.env.EXAMPLE_FAILED_TESTS.split(',') : [],
    ran: Number(process.env.EXAMPLE_TESTS_RAN ?? 3),
  };
  const decision = await checkCompletion(createHostedClient(), {
    task: 'Add a refund-eligibility check that refuses when tests fail.',
    output: 'Implemented combineDoneSignal; unit tests cover failed and passing suites.',
    acceptance: [
      'Failed tests prevent a done=true result',
      'Yes/no model checks are combined with AND',
    ],
    tests,
  });
  console.log(JSON.stringify(decision, null, 2));
}

if (process.argv[1] && import.meta.url.endsWith(process.argv[1].split(/[\\/]/).pop() ?? '')) {
  main().catch((error: unknown) => {
    console.error(error instanceof Error ? error.message : error);
    process.exitCode = 1;
  });
}

需要说明几点（均出自示例代码和注释）：

• System One 不跑测试，也没有 done 原语。 System One 不读取 CI，也不提供“完成”这个原语。测试结果是应用已有的确定性证据，示例里用 TestEvidence 传进来。
• 确定性证据优先。 即使模型给出的概率很高，combineDoneSignal 也会先看测试：一个测试都没跑时返回 no_tests_ran，跑了测试但没通过时返回 tests_failed。
• 两个是非判断用“且”组合。 两个概率都不低于 MIN_CRITERION_PROBABILITY = 0.7 才算 modelSaysDone。这个阈值由应用决定，示例注释写明它“不是产品默认值”，要在自己的案例上调。
• 托管模型：SYSTEM_ONE_MODEL 要从 GET /v1/models 获取。system-one.dev 提供 Jev（typesafe/jev）、Clef Flash（cloudflare/clef-flash）和 Clef（cloudflare/clef）模型服务。

动作表（中间区间重试、交人工、硬上限、停滞检测）见上文“动作表的写法”，与 AI SDK 的组合见“和 AI SDK stopWhen 怎么组合”。

接入 Claude Code（Stop hook）

开发团队的示例（examples/decision-articles/05-claude-code-stop-hook.ts）把上面的 checkCompletion() 包成一个 Claude Code 的 Stop hook。它从标准输入读取 hook 的 JSON，取其中的 last_assistant_message 作为 Agent 的输出，连同测试证据一起做完成判断。它的决定与上表基本一致，区别有两点：测试通过但是非检查没达标时，一律拦下、再跑一轮，不区分 0.4 以下交人工；也不做步数上限和停滞检测。它有三种输出：

• 阻止，再跑一轮：测试失败且模型也认为没完成、一个测试都没跑，或测试通过但是非检查低于阈值时，输出 {"decision": "block", "reason": "..."}，把原因告诉 Claude，让它继续工作。
• 交给人工：测试失败但模型认为已完成时，或者 stop_hook_active 为 true 而测试仍失败、一个测试都没跑时，不再阻止，而是输出 {"systemMessage": "Tests failing but agent believes it is done — needs human review"}，同时把这句话写到 stderr，让 Claude 停下，由人来看。官方 hooks 文档把 systemMessage 定义为向用户显示的警告信息。示例在所有交人工的情况下都输出这同一句提示，接入时建议按原因改写。
• 放行：测试通过且两个是非检查都达到阈值时，什么都不输出，Claude 可以停下。

输入里的 stop_hook_active 为 true，说明 Claude 已经是因为上一次被阻止才继续工作的。这时示例仍然先看测试：测试失败或一个测试都没跑，都走“交给人工”，不再阻止；测试通过就放行，不再看是非检查。也就是说，示例最多只阻止一次，第二次就交给人，而不是靠 Claude Code 自己的上限来结束循环。Claude Code 的官方 hooks 文档写明，Stop hook 连续让回合继续 8 次后，Claude Code 会忽略下一次阻止并结束回合；但这只是兜底，被强行结束的回合没有人接手，所以示例不依赖它。示例注释写明，这个接口契约是按 Claude Code 的 hooks 文档（2026-10-11）核对的；System One 本身没有 Claude Code 集成，也没有 done 原语。

ts/**
 * Article 5 — Claude Code Stop hook around System One completion checks.
 *
 * Contract verified against https://code.claude.com/docs/en/hooks (2026-10-11):
 *   Input (stdin JSON), Stop-specific: hook_event_name, stop_hook_active,
 *   last_assistant_message, plus common session_id / transcript_path / cwd /
 *   permission_mode. Use last_assistant_message; do not parse transcript_path.
 *   `stop_hook_active` is true when Claude is already continuing because a
 *   previous Stop hook blocked. This file still reads test results: failed
 *   tests hand off to a person (allow stop + `systemMessage` / stderr) so the
 *   hook cannot loop; passing tests allow stop. Claude Code also applies an
 *   8-consecutive-continuation cap.
 *   Output: omit JSON to allow stop; or
 *     { "decision": "block", "reason": "..." }
 *   or, when handing off without blocking:
 *     { "systemMessage": "..." }
 *   `reason` is required when decision is "block". Exit 0 with that JSON.
 *   Exit 2 + stderr also blocks; this file uses the JSON form.
 *   hookSpecificOutput.additionalContext is the non-error continue path —
 *   this hook uses decision/reason when Claude should keep working.
 *
 * System One still has no Claude Code integration or `done` primitive. Test
 * results are application evidence. Merge 05-claude-code-stop-hook.settings.json
 * into project `.claude/settings.json`.
 */
import type { EvaluationClient } from '@system-one-ai/core';
import {
  type TestEvidence,
  checkCompletion,
  createHostedClient,
} from './05-agent-completion-check.ts';

/** Documented Stop input fields used by this hook. Extra keys are ignored. */
export interface StopHookInput {
  hook_event_name?: string;
  session_id?: string;
  transcript_path?: string;
  cwd?: string;
  permission_mode?: string;
  stop_hook_active?: boolean;
  last_assistant_message?: string;
}

/** Documented Stop block output. The only `decision` value is `"block"`. */
export interface StopHookBlockOutput {
  decision: 'block';
  reason: string;
}

/** Allow stop and show a user-visible note. No `decision`, so Claude may stop. */
export interface StopHookHumanHandoff {
  systemMessage: string;
}

export type StopHookResult = StopHookBlockOutput | StopHookHumanHandoff | undefined;

/** Exact copy for the tests-failed / model-says-done handoff. */
export const HUMAN_REVIEW_NOTE = 'Tests failing but agent believes it is done — needs human review';

export function isStopHookHumanHandoff(result: StopHookResult): result is StopHookHumanHandoff {
  return Boolean(result && 'systemMessage' in result && !('decision' in result));
}

export function parseStopHookInput(raw: unknown): StopHookInput {
  if (typeof raw !== 'object' || raw === null || Array.isArray(raw)) {
    throw new Error('Stop hook stdin must be a JSON object.');
  }
  const record = raw as Record<string, unknown>;
  const text = (key: string) => {
    const value = record[key];
    return typeof value === 'string' ? value : undefined;
  };
  return {
    hook_event_name: text('hook_event_name'),
    session_id: text('session_id'),
    transcript_path: text('transcript_path'),
    cwd: text('cwd'),
    permission_mode: text('permission_mode'),
    stop_hook_active: typeof record.stop_hook_active === 'boolean' ? record.stop_hook_active : undefined,
    last_assistant_message: text('last_assistant_message'),
  };
}

export function readTestEvidenceFromEnv(env: NodeJS.ProcessEnv = process.env): TestEvidence {
  return {
    passed: env.EXAMPLE_TESTS_PASSED !== '0',
    failed: env.EXAMPLE_FAILED_TESTS ? env.EXAMPLE_FAILED_TESTS.split(',').filter(Boolean) : [],
    ran: Number(env.EXAMPLE_TESTS_RAN ?? 0),
  };
}

export function blockReasonFromCompletion(result: {
  done: boolean;
  reason: string;
  failed?: readonly string[];
  probabilities?: { acceptanceCovered: number; noOpenQuestions: number };
}) {
  if (result.reason === 'tests_failed') {
    const failed = result.failed?.length ? result.failed.join(', ') : 'unspecified tests';
    return `Stop blocked: tests failed (${failed}). Fix the failures before finishing.`;
  }
  if (result.reason === 'no_tests_ran') {
    return 'Stop blocked: no tests ran. Run the test suite and try again.';
  }
  if (result.reason === 'criteria_below_threshold') {
    const covered = result.probabilities?.acceptanceCovered;
    const open = result.probabilities?.noOpenQuestions;
    return `Stop blocked: yes/no completion checks are below threshold (acceptanceCovered=${covered}, noOpenQuestions=${open}). Address remaining criteria or questions.`;
  }
  return `Stop blocked: ${result.reason}.`;
}

/**
 * Map tests + model signal onto the documented Stop decision.
 *
 * (a) Tests failed and the model does not say done → block, another round.
 * (b) Tests failed but the model says done → allow stop + human-review note.
 * (c) No tests ran → block, another round.
 * When `stop_hook_active` is true, failed tests or zero runs hand off (b)
 * so the hook cannot loop; passing tests allow stop.
 */
export function decideStopHook(
  input: StopHookInput,
  checked: {
    done: boolean;
    reason: string;
    modelSaysDone: boolean;
    failed?: readonly string[];
    probabilities?: { acceptanceCovered: number; noOpenQuestions: number };
  },
  tests: TestEvidence,
): StopHookResult {
  if (input.hook_event_name !== undefined && input.hook_event_name !== 'Stop') {
    return undefined;
  }

  const noTestsRan = tests.ran === 0;
  const testsFailed = tests.ran > 0 && !tests.passed;

  if (testsFailed) {
    if (input.stop_hook_active === true || checked.modelSaysDone) {
      return { systemMessage: HUMAN_REVIEW_NOTE };
    }
    return { decision: 'block', reason: blockReasonFromCompletion({ ...checked, reason: 'tests_failed' }) };
  }

  if (noTestsRan) {
    if (input.stop_hook_active === true) {
      return { systemMessage: HUMAN_REVIEW_NOTE };
    }
    return { decision: 'block', reason: blockReasonFromCompletion({ ...checked, reason: 'no_tests_ran' }) };
  }

  if (input.stop_hook_active === true || checked.done) {
    return undefined;
  }
  return { decision: 'block', reason: blockReasonFromCompletion(checked) };
}

export function stopHookOutputFromCompletion(result: {
  done: boolean;
  reason: string;
  modelSaysDone?: boolean;
  failed?: readonly string[];
  probabilities?: { acceptanceCovered: number; noOpenQuestions: number };
}): StopHookResult {
  return decideStopHook(
    { hook_event_name: 'Stop', stop_hook_active: false },
    { ...result, modelSaysDone: result.modelSaysDone ?? false },
    {
      passed: result.reason !== 'tests_failed' && result.reason !== 'no_tests_ran',
      failed: result.failed ?? [],
      ran: result.reason === 'no_tests_ran' ? 0 : 1,
    },
  );
}

export async function evaluateStopHook(
  input: StopHookInput,
  client: EvaluationClient,
  context: {
    task: string;
    acceptance: readonly string[];
    tests: TestEvidence;
  },
): Promise<StopHookResult> {
  if (input.hook_event_name !== undefined && input.hook_event_name !== 'Stop') {
    return undefined;
  }
  const checked = await checkCompletion(client, {
    task: context.task,
    output: input.last_assistant_message ?? '',
    acceptance: context.acceptance,
    tests: context.tests,
  });
  return decideStopHook(input, checked, context.tests);
}

async function readStdin() {
  const chunks: Buffer[] = [];
  for await (const chunk of process.stdin) chunks.push(Buffer.from(chunk));
  const text = Buffer.concat(chunks).toString('utf8').trim();
  return text === '' ? {} : JSON.parse(text) as unknown;
}

async function main() {
  const input = parseStopHookInput(await readStdin());
  const output = await evaluateStopHook(input, createHostedClient(), {
    task: process.env.EXAMPLE_TASK ?? 'Complete the requested work and leave no blocking questions.',
    acceptance: process.env.EXAMPLE_ACCEPTANCE
      ? process.env.EXAMPLE_ACCEPTANCE.split('|').filter(Boolean)
      : ['Acceptance criteria are covered', 'No blocking questions remain'],
    tests: readTestEvidenceFromEnv(),
  });
  if (isStopHookHumanHandoff(output)) {
    console.error(output.systemMessage);
    process.stdout.write(`${JSON.stringify(output)}\n`);
    return;
  }
  if (output) process.stdout.write(`${JSON.stringify(output)}\n`);
}

if (process.argv[1] && import.meta.url.endsWith(process.argv[1].split(/[\\/]/).pop() ?? '')) {
  main().catch((error: unknown) => {
    console.error(error instanceof Error ? error.message : error);
    process.exitCode = 1;
  });
}

再把下面的配置（examples/decision-articles/05-claude-code-stop-hook.settings.json）合并进项目的 .claude/settings.json：

json{
  "hooks": {
    "Stop": [
      {
        "hooks": [
          {
            "type": "command",
            "command": "node",
            "args": [
              "--experimental-strip-types",
              "${CLAUDE_PROJECT_DIR}/examples/decision-articles/05-claude-code-stop-hook.ts"
            ]
          }
        ]
      }
    ]
  }
}

这个 hook 和 /goal 是两条不同的路：/goal 用 Claude Code 自带的评估器，Stop hook 让你用自己的测试证据和 System One 的是非概率来决定能不能停。

这些机制可以叠加使用：硬上限兜底，确定性检查优先，语义判断补足测试覆盖不到的地方。本站没有做过不同方案之间的实测对比，所以不给出成本或准确率数字。

延伸阅读

• 决策模型是什么？
• Jev 开源替代有哪些？
• 用 Score 替代 LLM 当裁判
• Agent 工具太多总选错？用 Choice 做工具路由
• System One 决策原语

---