---
title: "Agent 工具太多总选错？用 Choice 做工具路由（TypeScript 实战）"
description: "工具一多，Agent 就容易选错，还浪费上下文。本文讲清检索、懒加载和决策模型各管哪一步，并用 TypeScript 实现带置信度阈值的工具路由。"
canonical: "https://system-one.dev/zh-CN/blog/agent-tool-routing"
language: "zh-CN"
---

# Agent 工具太多总选错？用 Choice 做工具路由（TypeScript 实战）

Agent 工具太多选错怎么办？先用检索、懒加载或一份精简的白名单，把候选工具缩小到少数几个，再用一个有边界的选择步骤从中挑出最合适的工具，或判断不需要调用工具。用决策模型的 Choice 做这一步，能拿到每个选项的概率和置信度：置信度高时交给应用执行，置信度低时交回主模型或询问用户。选工具和生成参数是两件事，应该分开处理，也分开度量。

决策模型和 Choice / Score / Noul 三种原语的基础知识见 [决策模型是什么？](https://system-one.dev/zh-CN/blog/what-is-a-decision-model)；怎么在不同模型服务之间切换见 [Jev 开源替代有哪些？](https://system-one.dev/zh-CN/blog/jev-open-source-alternative)。

## 为什么工具越多，Agent 越容易选错

接了几个 MCP 服务器之后，工具定义很快就会塞满上下文。Anthropic 的工具搜索文档描述了两个问题：

- **上下文膨胀**：一个常见的多服务器配置（GitHub、Slack、Sentry、Grafana、Splunk）在 Claude 开始工作之前，光工具定义就可能占用约 5.5 万个 token。
- **选择准确率下降**：工具数超过 30 到 50 个之后，Claude 选对工具的能力就会下降。

这是 Anthropic 对自家模型的说法，其他模型的拐点不一定相同。原因不难理解：候选越多，描述相近的工具就越多，比如几个都叫“搜索”、几个都能“发消息”，模型也更容易被无关的定义分散注意力。开发者社区的抱怨也很直接：“too many tools and no clarity on which to use”，意思是工具太多，不知道该用哪个（[原帖](https://x.com/TryArcade/status/2098110514148622752)）。

## 三层架构：检索 → 选择 → 执行

社区目前的解法大多集中在第一层：先检索、懒加载、改写工具描述。把它们和后面两步放在一起看，工具路由可以分成三层：

1. **检索：缩小候选。** 用关键词、BM25、向量检索，或者模型自己调用的“工具搜索”，把成百上千个工具缩小到少数几个。Anthropic 的工具搜索默认每次返回最多 5 个工具，并建议把最常用的 3 到 5 个工具一直加载着，其余的延迟加载。
2. **选择：从候选中定一个。** 这是一次有边界的选择：答案只能是候选工具之一，或者“都不合适”。这一步正好是决策模型 Choice 的用武之地。
3. **执行：生成参数并调用。** 由应用的权限检查和主模型（或代码）负责生成参数，再调用工具。System One 不会替你调用工具，也不生成参数。

需要注意的是，检索这一层失败了，后面两层都救不回来：正确的工具没出现在候选里，再好的选择器也选不到它。第三方的对比测试（Stacklok 博客，厂商自测）也是这样解释工具选择准确率差距的。所以决策模型**不能代替检索**，它只负责检索之后的那一步。

下图是开发团队提供的三层架构示意图（`examples/decision-articles/04-three-tier-architecture.mmd`；同目录下还有渲染好的 `04-three-tier-architecture.svg`）。检索由应用负责，Choice 只给出工具名，置信度分档是应用自己设的阈值，不是托管服务的默认值。

```mermaid
%% Article 4 — three-tier tool routing after Choice.
%% Retrieve is application-owned. Choice only names a tool.
%% Confidence bands are application thresholds, not a hosted default.
flowchart TB
  subgraph retrieve [Layer 1 - Retrieve]
    Message[User message] --> Catalog[Allowlisted catalog]
    Catalog --> Candidates[Candidate tools]
  end

  subgraph select [Layer 2 - Select]
    Candidates --> Choice[System One Choice]
    Choice --> Band{Confidence band}
  end

  subgraph act [Layer 3 - Act]
    Band -->|high| Execute[Propose execute tool]
    Band -->|medium| Confirm[Ask user to confirm]
    Band -->|low or missing| Fallback[Main application LLM]
    Confirm -->|approved| Execute
    Confirm -->|declined| Stop[Do not execute]
  end
```

## 用 Choice 做工具选择

Choice 就是从一组具名选项里选一个。System One 的[决策原语文档](https://docs.system-one.dev/zh-CN/docs/primitives)写明，原生 Choice 的答案包含选中的选项名（`choice`）、以选项名为键的概率分布（`probabilities`），以及独立报告的统计值 `confidence`。System One SDK 的写法见下文示例。

### 选项 = 工具名 + 一句“何时使用”

选项的键就是工具名，值写清“什么情况下选它”。System One 的接入文档建议：工具说明应解释何时选择该项，不要包含凭据，也不能用描述来授予权限。决策原语文档也要求保持各选项含义清晰，让选项彼此区分开。

### 留一条“都不合适”的出路

System One 决策原语文档建议，强制归类会误导时加入审核选项；接入文档也建议在适合时提供明确的 `think` 或 `review` 选项，让模型不必硬选一个工具。下文示例用的是 `defer_to_llm`（交回主模型），白名单中还有 `ask_user`（询问用户）和 `human_review`（人工审核）两个“非工具”选项。

### 选项上限 255 个，为什么还要先缩小候选

System One 文档规定，一个 Choice 问题最多 255 个选项。TypeSafe 甚至建议做分类时直接给模型完整列表，而不是缩短的候选列表，因为每多一个选项只多花几个 token。

工具路由有所不同。System One 的接入文档明确建议：**从一个小型白名单开始，不要把 Agent 能访问的所有工具都交给模型**，也不要让模型生成的名称、URL 或 shell 命令绕过白名单。原因有两个：工具调用会产生实际后果，候选集本身就是权限边界；而每个工具的“何时使用”往往比分类标签长得多。实际做法可以按候选数量来定：候选少时直接放进 Choice；工具多、描述长、权限各异时，先检索出候选再交给 Choice。哪种方式更准，要用你自己的评估集比较，本站没有做过实测。

## 置信度阈值与回退

拿到 Choice 结果后，应用按置信度决定下一步。TypeSafe 的置信度文档给了一个“三段式”的起点：

- **高置信**：自动执行。工具路由里，就是交给应用的权限检查和调度器。
- **中等置信**：谨慎处理，比如让用户确认、标记复核，或先收集更多信息。
- **低置信**：不要执行，转给人工、请用户澄清，或回退到别的系统（比如主模型）。

几个要点：

- **置信度不等于“选对的概率”。** System One 决策原语文档写明，`confidence` 不一定等于选中选项的概率，两者都不是通用动作阈值。TypeSafe 文档补充说明，TypeSafe 的 Choice 置信度是根据概率分布算出来的统计量，概率全部集中在一个选项上为 1，平均分散为 0。TypeSafe 还建议同时参考最高概率、第一名与第二名的概率比。下文示例的注释也写明：置信度是一项可选的服务端统计，不一定代表“选中工具的概率”，也不是通用阈值。
- **阈值随风险变化。** 只读的查询可以用较低的阈值；付款、改权限、对外发消息这类动作，应该要求更高的置信度，或者无论如何都要人工确认。System One 的 Agent 接入文档也要求，付款、访问权限变更和对外发送消息必须保留政策要求的确认。
- **缺失就是未解决。** System One 文档写道：答案缺失或请求失败表示未解决，不是自动的“是”“否”或最低分。示例在 `confidence` 缺失时一律不执行。
- **模型结果不是授权。** 模型选中了某个工具，不代表这个用户有权执行它。身份、权限和审批要在模型之外检查。
- **阈值从哪里来**：System One 文档写明，阈值应该来自你自己的评估案例和后果分析，而不是一个通用的百分比。换了模型、说明、选项或描述之后，都要重新评估。

### 三段式的写法

开发团队提供了一个三段式示例 `examples/decision-articles/04-three-tier-routing.ts`。它从主示例 `04-agent-tool-routing.ts` 导入白名单、检索函数和 `DEFER_TO_LLM`，主示例见下文“TypeScript 示例”一节。置信度不低于 `HIGH_EXECUTE_CONFIDENCE`（0.75）时返回“执行”；介于 `MEDIUM_CONFIRM_CONFIDENCE`（0.45）和高阈值之间时，调用应用自己的确认回调，由用户决定；更低、缺失，或模型选了 `defer_to_llm` 时，交回应用的主模型。两个阈值都是示例值，注释写明要按标注案例和动作风险调。

```ts
/**
 * Article 4 — Three-tier tool routing after Choice.
 *
 * TypeSafe's confidence guidance is a three-band starting point: high →
 * execute, medium → confirm with the user, low → do not execute. System One
 * only returns Choice + optional confidence. Thresholds, the confirm UI, and
 * the main-LLM fallback are application-owned. Missing confidence stays
 * unknown and is treated as "do not execute".
 *
 * This file does not generate tool arguments and does not call tools.
 */
import { choice } from '@system-one-ai/core';
import type { ChoiceAnswer, EvaluationClient } from '@system-one-ai/core';
import {
  DEFER_TO_LLM,
  TOOL_CATALOG,
  createHostedClient,
  retrieveCandidateTools,
} from './04-agent-tool-routing.ts';

/** Application-owned. Tune on labeled cases and action risk; not a hosted default. */
export const HIGH_EXECUTE_CONFIDENCE = 0.75;
export const MEDIUM_CONFIRM_CONFIDENCE = 0.45;

export type RouteTier = 'execute' | 'confirm' | 'fallback';

export function buildToolChoiceCriteria(
  tools: readonly { id: string; whenToUse: string }[],
) {
  return Object.fromEntries([
    ...tools.map(tool => [tool.id, tool.whenToUse]),
    [DEFER_TO_LLM, 'None of the allowlisted tools fits; continue with the main application LLM'],
  ]);
}

export function classifyRouteTier(answer: ChoiceAnswer<string>): RouteTier {
  if (answer.choice === DEFER_TO_LLM) return 'fallback';
  if (answer.confidence === undefined) return 'fallback';
  if (answer.confidence >= HIGH_EXECUTE_CONFIDENCE) return 'execute';
  if (answer.confidence >= MEDIUM_CONFIRM_CONFIDENCE) return 'confirm';
  return 'fallback';
}

export interface ConfirmToolInput {
  message: string;
  proposedTool: string;
  confidence: number;
}

/**
 * @param askUserToConfirm Your confirm UI or approval queue. Not a System One method.
 * @param completeWithApplicationLlm Your agent's existing completion path.
 */
export async function routeToolThreeTier(
  client: EvaluationClient,
  message: string,
  callbacks: {
    askUserToConfirm: (input: ConfirmToolInput) => Promise<boolean>;
    completeWithApplicationLlm: (prompt: string) => Promise<string>;
  },
) {
  const candidates = retrieveCandidateTools(message, TOOL_CATALOG);
  const result = await client.evaluate({
    state: { message, retrievedTools: candidates.map(tool => tool.id) },
    questions: {
      tool: choice('Choose the next allowed tool. Do not execute it.', buildToolChoiceCriteria(candidates)),
    },
  });
  const answer = result.answers.tool;
  const tier = classifyRouteTier(answer);

  if (tier === 'execute') {
    return {
      action: 'execute_tool' as const,
      tier,
      proposedTool: answer.choice,
      confidence: answer.confidence,
    };
  }

  if (tier === 'confirm') {
    const approved = await callbacks.askUserToConfirm({
      message,
      proposedTool: answer.choice,
      confidence: answer.confidence as number,
    });
    return approved
      ? {
          action: 'execute_tool' as const,
          tier,
          proposedTool: answer.choice,
          confidence: answer.confidence,
          confirmedByUser: true as const,
        }
      : {
          action: 'user_declined' as const,
          tier,
          proposedTool: answer.choice,
          confidence: answer.confidence,
        };
  }

  const fallback = await callbacks.completeWithApplicationLlm(message);
  return {
    action: 'fallback_llm' as const,
    tier,
    proposedTool: answer.choice,
    confidence: answer.confidence,
    fallback,
  };
}

async function main() {
  const message = process.env.EXAMPLE_MESSAGE ?? 'Can you explain how credits are calculated?';
  const decision = await routeToolThreeTier(createHostedClient(), message, {
    askUserToConfirm: async () => {
      throw new Error('No System One confirm UI. Wire your approval callback here.');
    },
    completeWithApplicationLlm: async () => {
      throw new Error('No System One LLM-completion endpoint. Wire your existing model call here.');
    },
  });
  console.log(JSON.stringify(decision, null, 2));
}

if (process.argv[1] && import.meta.url.endsWith(process.argv[1].split(/[\\/]/).pop() ?? '')) {
  main().catch((error: unknown) => {
    console.error(error instanceof Error ? error.message : error);
    process.exitCode = 1;
  });
}
```

## 度量：选择准确率和参数有效率要分开算

工具调用失败有两种不同的情况：一是**选错了工具**，二是**选对了工具但参数错了**。开发者社区也有人提出，Agent 可能选错工具，也可能选对工具却给了错误参数，所以工具选择和参数有效性应该分开度量（[原帖](https://x.com/SabarnathX/status/2108198226742366460)）。在本文的三层架构里，这两种错误分别出在第 2 层和第 3 层，所以建议分开统计：

- **选择准确率**：在带标注的评估集上，Choice 选出的工具与预期工具一致的比例。除了总体准确率，还要单独看“该转人工却没转”的漏判。System One 文档建议分别统计错误工具选择和遗漏审核的情况，并加入模糊请求，以及试图强制模型选择其他工具的指令。
- **检索召回率**：正确工具出现在候选中的比例。它决定了选择准确率的上限。
- **参数有效率**：在选对工具的前提下，参数通过校验、调用成功的比例。这属于主模型或代码的职责，与决策模型无关。
- **回退率**：被交回主模型、用户或人工的比例。它直接影响成本和用户体验。

System One 文档还提醒：HTTP 请求成功、模型给出较高概率，都不能证明路由是安全的。应该先以“只提供建议”的方式运行，再根据已有政策决定哪些只读动作可以自动执行。

### 参数生成：选完工具再单独生成、单独校验

下面的示例（`examples/decision-articles/04-generate-tool-params.ts`）在三段式路由判定“执行”之后，才调用应用自己的参数生成回调，并用 `paramsAreValid()` 检查必填参数是否齐全、非空。Choice 只负责选出工具名，System One 不生成、不校验参数，也不调用工具。示例中的必填字段表（`TOOL_PARAM_REQUIRED`）只是演示。

```ts
/**
 * Article 4 — Generate tool-call parameters after routing.
 *
 * Choice only names the tool. System One does not emit arguments, validate
 * them, or invoke the tool. Parameter generation stays in the application
 * (deterministic code or your main LLM). Measure parameter validity separately
 * from routing accuracy.
 */
import type { EvaluationClient } from '@system-one-ai/core';
import { DEFER_TO_LLM, createHostedClient } from './04-agent-tool-routing.ts';
import { routeToolThreeTier } from './04-three-tier-routing.ts';

export interface GeneratedToolParams {
  readonly [key: string]: string;
}

export const TOOL_PARAM_REQUIRED: Readonly<Record<string, readonly string[]>> = {
  search_docs: ['query'],
  ask_user: ['question'],
  human_review: ['reason'],
};

export function paramsAreValid(toolId: string, params: unknown): boolean {
  const required = TOOL_PARAM_REQUIRED[toolId];
  if (!required) return false;
  if (typeof params !== 'object' || params === null || Array.isArray(params)) return false;
  const record = params as Record<string, unknown>;
  return required.every(key => typeof record[key] === 'string' && record[key].trim().length > 0);
}

export interface GenerateParamsInput {
  message: string;
  toolId: string;
}

/**
 * @param generateParams Application-owned. Not a System One method.
 *        The decision model has already picked `toolId` and must not invent args.
 */
export async function routeThenGenerateParams(
  client: EvaluationClient,
  message: string,
  generateParams: (input: GenerateParamsInput) => Promise<unknown>,
  callbacks: {
    askUserToConfirm: (input: { message: string; proposedTool: string; confidence: number }) => Promise<boolean>;
    completeWithApplicationLlm: (prompt: string) => Promise<string>;
  },
) {
  const routed = await routeToolThreeTier(client, message, callbacks);
  if (routed.action !== 'execute_tool' || routed.proposedTool === DEFER_TO_LLM) {
    return { ...routed, params: undefined, paramsValid: false as const };
  }
  const params = await generateParams({ message, toolId: routed.proposedTool });
  return {
    ...routed,
    params,
    paramsValid: paramsAreValid(routed.proposedTool, params),
  };
}

async function main() {
  const message = process.env.EXAMPLE_MESSAGE ?? 'Can you explain how credits are calculated?';
  const decision = await routeThenGenerateParams(
    createHostedClient(),
    message,
    async ({ message: userMessage, toolId }) => {
      if (toolId === 'search_docs') return { query: userMessage };
      throw new Error(`No application parameter builder wired for ${toolId}.`);
    },
    {
      askUserToConfirm: async () => true,
      completeWithApplicationLlm: async () => {
        throw new Error('No System One LLM-completion endpoint. Wire your existing model call here.');
      },
    },
  );
  console.log(JSON.stringify(decision, null, 2));
}

if (process.argv[1] && import.meta.url.endsWith(process.argv[1].split(/[\\/]/).pop() ?? '')) {
  main().catch((error: unknown) => {
    console.error(error instanceof Error ? error.message : error);
    process.exitCode = 1;
  });
}
```

### 离线评估：检索召回、选择准确率、漏判率、参数有效率

下面的示例（`examples/decision-articles/04-routing-eval-harness.ts`）在一组带标注的案例上同时算出四个指标：检索召回率、Choice 选择准确率、该转人工却没转的漏判率，以及“选对工具的前提下”的参数有效率。System One 没有评估数据集或裁判 API，这些都由这段应用代码完成。示例里的 4 条案例（`ROUTE_EVAL_CASES`）只用于演示，注释写明调阈值前要换成你自己的案例。

```ts
/**
 * Article 4 — Offline routing-accuracy harness on a labeled set.
 *
 * System One has no eval-dataset or judge API. This file scores:
 *   1. retrieval recall — expected tool present in the candidate set
 *   2. choice accuracy — Choice equals the labeled tool
 *   3. missed-review rate — labeled human_review but Choice picked something else
 * Parameter validity is a separate column, computed only on correctly routed
 * rows, because argument generation is not a decision-model job.
 */
import { choice } from '@system-one-ai/core';
import type { EvaluationClient } from '@system-one-ai/core';
import {
  DEFER_TO_LLM,
  TOOL_CATALOG,
  createHostedClient,
  retrieveCandidateTools,
} from './04-agent-tool-routing.ts';
import { paramsAreValid } from './04-generate-tool-params.ts';
import { buildToolChoiceCriteria } from './04-three-tier-routing.ts';

export interface LabeledRouteCase {
  id: string;
  message: string;
  expectedTool: string;
  /** When set, used only after a correct Choice to score argument generation. */
  expectedParams?: Readonly<Record<string, string>>;
}

/** Small labeled set for the article. Replace with your own cases before tuning thresholds. */
export const ROUTE_EVAL_CASES: readonly LabeledRouteCase[] = [
  { id: 'credits', message: 'How are credits calculated?', expectedTool: 'search_docs' },
  { id: 'vague', message: 'It broke and does not work', expectedTool: 'ask_user' },
  { id: 'refund', message: 'I was charged twice and want a refund', expectedTool: 'human_review' },
  { id: 'unrelated', message: 'Write a haiku about rain', expectedTool: DEFER_TO_LLM },
];

export interface RouteEvalRow {
  id: string;
  expectedTool: string;
  predictedTool: string | undefined;
  retrieved: readonly string[];
  retrievedExpected: boolean;
  correctChoice: boolean;
  missedReview: boolean;
  paramsValid: boolean | undefined;
}

export function evaluateRetrieval(cases: readonly LabeledRouteCase[] = ROUTE_EVAL_CASES) {
  return cases.map((labeled) => {
    const retrieved = retrieveCandidateTools(labeled.message, TOOL_CATALOG).map(tool => tool.id);
    const retrievedExpected =
      labeled.expectedTool === DEFER_TO_LLM || retrieved.includes(labeled.expectedTool);
    return { id: labeled.id, retrieved, retrievedExpected };
  });
}

export async function evaluateRoutingAccuracy(
  client: EvaluationClient,
  cases: readonly LabeledRouteCase[] = ROUTE_EVAL_CASES,
  generateParams?: (input: { message: string; toolId: string }) => Promise<unknown>,
) {
  const rows: RouteEvalRow[] = [];
  for (const labeled of cases) {
    const candidates = retrieveCandidateTools(labeled.message, TOOL_CATALOG);
    const retrieved = candidates.map(tool => tool.id);
    const retrievedExpected =
      labeled.expectedTool === DEFER_TO_LLM || retrieved.includes(labeled.expectedTool);
    const result = await client.evaluate({
      state: { message: labeled.message, retrievedTools: retrieved },
      questions: {
        tool: choice('Choose the next allowed tool. Do not execute it.', buildToolChoiceCriteria(candidates)),
      },
    });
    const predictedTool = result.answers.tool.choice;
    const correctChoice = predictedTool === labeled.expectedTool;
    const missedReview = labeled.expectedTool === 'human_review' && predictedTool !== 'human_review';

    let paramsValid: boolean | undefined;
    if (correctChoice && labeled.expectedParams && generateParams) {
      const params = await generateParams({ message: labeled.message, toolId: predictedTool });
      paramsValid = paramsAreValid(predictedTool, params);
    }

    rows.push({
      id: labeled.id,
      expectedTool: labeled.expectedTool,
      predictedTool,
      retrieved,
      retrievedExpected,
      correctChoice,
      missedReview,
      paramsValid,
    });
  }
  return rows;
}

export function summarizeRouteEval(rows: readonly RouteEvalRow[]) {
  const reviewCases = rows.filter(row => row.expectedTool === 'human_review');
  const correctWithParams = rows.filter(row => row.correctChoice && row.paramsValid !== undefined);
  return {
    n: rows.length,
    retrievalRecall: rows.length === 0 ? 0 : rows.filter(row => row.retrievedExpected).length / rows.length,
    choiceAccuracy: rows.length === 0 ? 0 : rows.filter(row => row.correctChoice).length / rows.length,
    missedReviewRate: reviewCases.length === 0
      ? 0
      : reviewCases.filter(row => row.missedReview).length / reviewCases.length,
    paramValidityAmongCorrect: correctWithParams.length === 0
      ? undefined
      : correctWithParams.filter(row => row.paramsValid).length / correctWithParams.length,
  };
}

async function main() {
  const retrieval = evaluateRetrieval();
  console.log(JSON.stringify({ retrieval, retrievalOnly: true }, null, 2));
  if (!process.env.SYSTEM_ONE_API_KEY) return;
  const rows = await evaluateRoutingAccuracy(createHostedClient());
  console.log(JSON.stringify({ rows, summary: summarizeRouteEval(rows) }, null, 2));
}

if (process.argv[1] && import.meta.url.endsWith(process.argv[1].split(/[\\/]/).pop() ?? '')) {
  main().catch((error: unknown) => {
    console.error(error instanceof Error ? error.message : error);
    process.exitCode = 1;
  });
}
```

## 工具路由和模型路由有什么区别

两者都是从预设选项中做一次选择，都可以用决策模型实现，但选项不同、指标也不同：

- **工具路由**决定这一步调用哪个工具（或者不调用）。主要指标是选择准确率、漏判率和参数有效率。
- **模型路由**决定把请求交给哪个模型处理。Microsoft 在 Microsoft-Decision-1 的发布博客中把它描述为“按质量、成本和延迟要求选出最合适的模型”（厂商说法），主要指标是质量、成本和延迟。

不少决策模型厂商都把“路由”列为主要用途：Microsoft 称 Microsoft-Decision-1 适用于 routing、classification 等场景，RouterPlus 的文档也说它的 `/v1/decisions` 可用于 routing、classification、moderation（均为厂商说法）。选哪家模型、怎么在不同服务之间切换，可以参考 [Jev 开源替代有哪些？2026 决策模型选型地图与 TypeScript 方案](https://system-one.dev/zh-CN/blog/jev-open-source-alternative)。如果要把决策模型当成评估环节的第一道关，参见 [用 Score 替代 LLM 当裁判](https://system-one.dev/zh-CN/blog/llm-as-a-judge-alternative)。

按难度把请求分给小模型或大模型的完整做法，详见 [模型路由：先判断难度，再分给小模型或大模型](https://system-one.dev/zh-CN/blog/model-routing)。

## TypeScript 示例：检索候选 → Choice → 低置信回退

**在线试一下**：[NPC 下一步动作](https://system-one.dev/zh-CN/demo/npc-action)（示例结果，登录后可跑真实数据）

下面这段代码来自开发团队提供的示例（`examples/decision-articles/04-agent-tool-routing.ts`）。它用关键词从一份静态白名单中检索候选工具，再用 Choice 选一个，并加入一个 `defer_to_llm` 选项。只有当置信度存在且不低于阈值时，才返回建议调用的工具；否则交回应用自己的主模型。

```ts
/**
 * Article 4 — Agent tool routing: retrieve candidates, Choice, fall back on low confidence.
 *
 * System One can pick one named option from the set you send. It does not
 * retrieve tools, execute them, or run a general LLM completion. Retrieval
 * and the "main LLM" fallback are application functions in this file.
 *
 * Confidence is an optional provider statistic, not necessarily P(selected
 * tool) and not a universal threshold. Missing confidence stays unknown —
 * this example treats unknown as "do not execute".
 */
import { createSystemOne, choice } from '@system-one-ai/core';
import type { ChoiceAnswer, EvaluationClient } from '@system-one-ai/core';
import { systemOneAdapter } from '@system-one-ai/adapter-system-one';
import { createFetchTransport } from '@system-one-ai/transport-fetch';

export interface ToolSpec {
  id: string;
  whenToUse: string;
  keywords: readonly string[];
}

/** Static allowlist. There is no System One tool-catalog endpoint. */
export const TOOL_CATALOG: readonly ToolSpec[] = [
  { id: 'search_docs', whenToUse: 'A product or usage question can be answered from documentation', keywords: ['credit', 'docs', 'how', 'price'] },
  { id: 'ask_user', whenToUse: 'Essential details are missing; ask before proceeding', keywords: ['it broke', 'does not work', 'help'] },
  { id: 'human_review', whenToUse: 'A payment dispute, refund, or policy exception needs a person', keywords: ['refund', 'charged', 'dispute'] },
];

export const DEFER_TO_LLM = 'defer_to_llm';
export const MIN_EXECUTE_CONFIDENCE = 0.55;

export function retrieveCandidateTools(message: string, catalog: readonly ToolSpec[] = TOOL_CATALOG) {
  const text = message.toLowerCase();
  const hits = catalog.filter(tool => tool.keywords.some(keyword => text.includes(keyword)));
  return hits.length > 0 ? hits : catalog.filter(tool => tool.id === 'ask_user' || tool.id === 'human_review');
}

export function shouldExecuteChosenTool(answer: ChoiceAnswer<string>) {
  if (answer.choice === DEFER_TO_LLM) return false;
  if (answer.confidence === undefined) return false;
  return answer.confidence >= MIN_EXECUTE_CONFIDENCE;
}

export function createHostedClient() {
  const apiKey = process.env.SYSTEM_ONE_API_KEY;
  const baseURL = process.env.SYSTEM_ONE_BASE_URL;
  const model = process.env.SYSTEM_ONE_MODEL;
  if (!apiKey || !baseURL || !model) {
    throw new Error('Set SYSTEM_ONE_API_KEY, SYSTEM_ONE_BASE_URL (…/v1), and SYSTEM_ONE_MODEL from GET /v1/models.');
  }
  return createSystemOne({
    apiKey, baseURL, model, adapter: systemOneAdapter,
    transport: createFetchTransport(), timeoutMs: 30_000, maxRetries: 0,
  });
}

/**
 * @param completeWithApplicationLlm Your agent's existing completion path.
 *        System One has no chat/completions URL for this fallback.
 */
export async function routeTool(
  client: EvaluationClient,
  message: string,
  completeWithApplicationLlm: (prompt: string) => Promise<string>,
) {
  const candidates = retrieveCandidateTools(message);
  const criteria = Object.fromEntries([
    ...candidates.map(tool => [tool.id, tool.whenToUse]),
    [DEFER_TO_LLM, 'None of the allowlisted tools fits; continue with the main application LLM'],
  ]);
  const result = await client.evaluate({
    state: { message, retrievedTools: candidates.map(tool => tool.id) },
    questions: {
      tool: choice('Choose the next allowed tool. Do not execute it.', criteria),
    },
  });
  const answer = result.answers.tool;
  if (!shouldExecuteChosenTool(answer)) {
    const fallback = await completeWithApplicationLlm(message);
    return {
      action: 'fallback_llm' as const,
      proposedTool: answer.choice,
      confidence: answer.confidence,
      fallback,
    };
  }
  return {
    action: 'propose_tool' as const,
    proposedTool: answer.choice,
    confidence: answer.confidence,
  };
}

async function main() {
  const message = process.env.EXAMPLE_MESSAGE ?? 'Can you explain how credits are calculated?';
  const decision = await routeTool(createHostedClient(), message, async () => {
    throw new Error('No System One LLM-completion endpoint. Wire your existing model call here.');
  });
  console.log(JSON.stringify(decision, null, 2));
}

if (process.argv[1] && import.meta.url.endsWith(process.argv[1].split(/[\\/]/).pop() ?? '')) {
  main().catch((error: unknown) => {
    console.error(error instanceof Error ? error.message : error);
    process.exitCode = 1;
  });
}
```

需要说明几点（均出自示例代码和注释）：

- **检索和回退都是应用自己的函数。** System One 不检索工具、不执行工具，也不提供通用的 LLM 补全接口；也没有工具目录接口，`TOOL_CATALOG` 是代码里的静态白名单。示例中的关键词匹配只是演示用的占位实现，生产环境可以换成 BM25 或向量检索。
- **只建议，不执行。** 置信度足够时，示例返回 `action: 'propose_tool'`，把是否执行交给你的权限检查和调度器。
- **阈值是示例值。** `MIN_EXECUTE_CONFIDENCE = 0.55` 只是示例，需要按上文的方法在自己的评估集上重新选。置信度缺失（`undefined`）时，示例按“不执行”处理。
- **托管模型**：`SYSTEM_ONE_MODEL` 要从 `GET /v1/models` 获取。system-one.dev 提供 Jev（`typesafe/jev`）、Clef Flash（`cloudflare/clef-flash`）和 Clef（`cloudflare/clef`）模型服务。

上文“三段式的写法”和“度量”两节中的示例补上了询问用户的中间档、参数生成和离线评估。下面两个示例演示如何接入 Vercel AI SDK 和 MCP。

### 接入 Vercel AI SDK

这个示例（`examples/decision-articles/04-vercel-ai-sdk-tools.ts`）用 AI SDK 的 `tool()` 定义白名单里的三个工具，先用三段式路由选出工具，再调用 `generateText`，通过 `activeTools` 和 `toolChoice` 把模型限定在这一个工具上，由 AI SDK 绑定的语言模型填写参数。示例注释写明：`@system-one-ai/adapter-vercel` 是 Vercel 的评估（Evaluation）适配器，不是 AI SDK 的工具适配器，也不是这个仓库的依赖，所以没有引入；传给 `generateText` 的 `model` 是你自己的模型服务，不是 System One。示例基于仓库使用的 `ai@6.0.82`。

```ts
/**
 * Article 4 — Vercel AI SDK tools wrapping System One routing.
 *
 * `@system-one-ai/adapter-vercel` is a published Vercel *Evaluation* adapter
 * (not an AI SDK tools adapter) and is not a dependency of this repo. This
 * file uses the already-installed `ai` package: `tool()`, `generateText`,
 * `activeTools`, and `toolChoice: { type: 'tool', toolName }`.
 *
 * Choice still only picks the tool. The language model bound to `generateText`
 * fills `inputSchema`. System One does not generate arguments.
 */
import { generateText, tool } from 'ai';
import type { LanguageModel, ToolSet } from 'ai';
import { z } from 'zod';
import type { EvaluationClient } from '@system-one-ai/core';
import { createHostedClient } from './04-agent-tool-routing.ts';
import { routeToolThreeTier } from './04-three-tier-routing.ts';

export const searchDocsTool = tool({
  description: 'A product or usage question can be answered from documentation',
  inputSchema: z.object({
    query: z.string().describe('Documentation search query'),
  }),
  execute: async ({ query }) => ({ source: 'application-docs-search', query }),
});

export const askUserTool = tool({
  description: 'Essential details are missing; ask before proceeding',
  inputSchema: z.object({
    question: z.string().describe('Clarifying question for the user'),
  }),
  execute: async ({ question }) => ({ source: 'application-ask-user', question }),
});

export const humanReviewTool = tool({
  description: 'A payment dispute, refund, or policy exception needs a person',
  inputSchema: z.object({
    reason: z.string().describe('Why a person must review this request'),
  }),
  execute: async ({ reason }) => ({ source: 'application-human-review', reason }),
});

export const allowlistedAiTools = {
  search_docs: searchDocsTool,
  ask_user: askUserTool,
  human_review: humanReviewTool,
} satisfies ToolSet;

export type AllowlistedToolName = keyof typeof allowlistedAiTools;

export function routedGenerateTextOptions(
  proposedTool: AllowlistedToolName,
  message: string,
) {
  return {
    prompt: message,
    tools: allowlistedAiTools,
    activeTools: [proposedTool] as Array<AllowlistedToolName>,
    toolChoice: { type: 'tool' as const, toolName: proposedTool },
  };
}

/**
 * Route with System One, then let the AI SDK model fill that tool's arguments.
 *
 * @param model A Vercel AI SDK `LanguageModel` from your existing provider
 *        package (for example `@ai-sdk/openai`). Not a System One model.
 */
export async function routeThenGenerateText(
  client: EvaluationClient,
  model: LanguageModel,
  message: string,
  callbacks: {
    askUserToConfirm: (input: { message: string; proposedTool: string; confidence: number }) => Promise<boolean>;
    completeWithApplicationLlm: (prompt: string) => Promise<string>;
  },
) {
  const routed = await routeToolThreeTier(client, message, callbacks);
  if (routed.action !== 'execute_tool' || !(routed.proposedTool in allowlistedAiTools)) {
    return { routed, generation: undefined };
  }
  const proposedTool = routed.proposedTool as AllowlistedToolName;
  const generation = await generateText({
    model,
    ...routedGenerateTextOptions(proposedTool, message),
  });
  return { routed, generation };
}

async function main() {
  const message = process.env.EXAMPLE_MESSAGE ?? 'Can you explain how credits are calculated?';
  const routed = await routeToolThreeTier(createHostedClient(), message, {
    askUserToConfirm: async () => true,
    completeWithApplicationLlm: async () => {
      throw new Error('No System One LLM-completion endpoint. Wire your existing model call here.');
    },
  });
  if (routed.action === 'execute_tool' && routed.proposedTool in allowlistedAiTools) {
    console.log(JSON.stringify({
      routed,
      generateTextOptions: routedGenerateTextOptions(routed.proposedTool as AllowlistedToolName, message),
    }, null, 2));
    return;
  }
  console.log(JSON.stringify({ routed }, null, 2));
}

if (process.argv[1] && import.meta.url.endsWith(process.argv[1].split(/[\\/]/).pop() ?? '')) {
  main().catch((error: unknown) => {
    console.error(error instanceof Error ? error.message : error);
    process.exitCode = 1;
  });
}
```

### 接入 MCP 工具

这个示例（`examples/decision-articles/04-mcp-tool-routing.ts`）只用官方 `@modelcontextprotocol/sdk` 的客户端类型：先用 `listTools()` 列出已连接 MCP 服务器上的工具，把工具名和描述转成 Choice 选项（加上 `defer_to_llm`），置信度达到高阈值且选中的工具确实在列表里时，才返回建议调用的工具和它的 `inputSchema`；真正调用由应用通过 `callTool()` 完成，参数来自你的代码或主模型。System One 本身不支持 MCP，也不检索、不执行工具。注意，这个示例把所有列出的 MCP 工具都交给 Choice，工具很多时，仍建议先按上文做检索。

```ts
/**
 * Article 4 — Route among tools advertised by an MCP client.
 *
 * Uses official `@modelcontextprotocol/sdk` client types only: `Client.listTools()`
 * and `Client.callTool({ name, arguments })`. System One does not speak MCP,
 * retrieve tools, or execute them. After Choice, the application calls MCP.
 */
import { choice } from '@system-one-ai/core';
import type { EvaluationClient } from '@system-one-ai/core';
import type { Client } from '@modelcontextprotocol/sdk/client';
import { DEFER_TO_LLM } from './04-agent-tool-routing.ts';
import { HIGH_EXECUTE_CONFIDENCE } from './04-three-tier-routing.ts';

export type McpToolLister = Pick<Client, 'listTools'>;
export type McpToolCaller = Pick<Client, 'callTool'>;
export type McpListedTool = Awaited<ReturnType<Client['listTools']>>['tools'][number];

export function mcpToolsToChoiceCriteria(tools: readonly McpListedTool[]) {
  return Object.fromEntries([
    ...tools.map(tool => [
      tool.name,
      tool.description ?? `MCP tool ${tool.name}. Use only when this tool is the best next action.`,
    ]),
    [DEFER_TO_LLM, 'None of the listed MCP tools fits; continue with the main application LLM'],
  ]);
}

export async function listMcpTools(mcp: McpToolLister) {
  const listed = await mcp.listTools();
  return listed.tools;
}

/**
 * List MCP tools, then let Choice pick one name. Does not call the tool.
 */
export async function routeMcpTool(
  client: EvaluationClient,
  mcp: McpToolLister,
  message: string,
) {
  const tools = await listMcpTools(mcp);
  if (tools.length === 0) {
    return {
      action: 'fallback_llm' as const,
      proposedTool: DEFER_TO_LLM,
      confidence: undefined,
      available: [] as string[],
    };
  }
  const result = await client.evaluate({
    state: { message, retrievedTools: tools.map(tool => tool.name) },
    questions: {
      tool: choice('Choose the next MCP tool. Do not execute it.', mcpToolsToChoiceCriteria(tools)),
    },
  });
  const answer = result.answers.tool;
  const available = tools.map(tool => tool.name);
  if (
    answer.choice === DEFER_TO_LLM
    || answer.confidence === undefined
    || answer.confidence < HIGH_EXECUTE_CONFIDENCE
    || !available.includes(answer.choice)
  ) {
    return {
      action: 'fallback_llm' as const,
      proposedTool: answer.choice,
      confidence: answer.confidence,
      available,
    };
  }
  return {
    action: 'propose_mcp_tool' as const,
    proposedTool: answer.choice,
    confidence: answer.confidence,
    available,
    inputSchema: tools.find(tool => tool.name === answer.choice)?.inputSchema,
  };
}

/**
 * Application-owned execution step. Arguments come from your code or main LLM,
 * not from System One.
 */
export async function callRoutedMcpTool(
  mcp: McpToolCaller,
  name: string,
  args: Record<string, unknown>,
) {
  return mcp.callTool({ name, arguments: args });
}

async function main() {
  throw new Error(
    'This file routes tools from an MCP Client you already connected. There is no System One MCP endpoint.',
  );
}

if (process.argv[1] && import.meta.url.endsWith(process.argv[1].split(/[\\/]/).pop() ?? '')) {
  main().catch((error: unknown) => {
    console.error(error instanceof Error ? error.message : error);
    process.exitCode = 1;
  });
}
```

更完整的鉴权、幂等和重试说明，见站内文档 [Agent 接入](https://docs.system-one.dev/zh-CN/docs/agent-integration)。
> 表中各方案可以组合使用。Anthropic 的数据针对的是 Claude，不代表其他模型的表现。本站没有做过同类实测。


## 各层方案分工表（核验日期 2026-10-11）

| 方案 | 解决哪一段 | 不解决什么 | 来源 |
| --- | --- | --- | --- |
| 精简白名单、改写工具描述<br><small>[System One Agent 接入文档](https://docs.system-one.dev/zh-CN/docs/agent-integration) · 2026-10-11</small> | 控制候选范围，让选项彼此区分开 | 工具多了之后仍需检索；描述不能授予权限<br><small>[System One Agent 接入文档](https://docs.system-one.dev/zh-CN/docs/agent-integration) · 2026-10-11</small> | 官方文档 |
| 工具检索 / 工具搜索<br><small>[Anthropic 工具搜索文档](https://console.anthropic.com/docs/en/agents-and-tools/tool-use/tool-search-tool) · 2026-10-11</small> | 从大量工具中找出少数候选（默认最多返回 5 个）<br><small>[Anthropic 工具搜索文档](https://console.anthropic.com/docs/en/agents-and-tools/tool-use/tool-search-tool) · 2026-10-11</small> | 会多一个检索步骤，带来额外延迟<br>检索失败时，后续步骤无法补救（第三方观点）；[Stacklok 博客](https://stacklok.com/blog/stackloks-mcp-optimizer-vs-anthropics-tool-search-tool-a-head-to-head-comparison/) · 2026-10-11<br><small>[Anthropic 工程博客](https://www.anthropic.com/engineering/advanced-tool-use) · 2026-10-11</small> | 官方文档（Anthropic，针对 Claude） |
| 懒加载（延迟加载工具定义）<br><small>[Anthropic 工程博客](https://www.anthropic.com/engineering/advanced-tool-use) · 2026-10-11</small> | 减少上下文中的工具定义；建议常驻 3 到 5 个最常用工具<br><small>[Anthropic 工程博客](https://www.anthropic.com/engineering/advanced-tool-use) · 2026-10-11</small> | 不负责在候选之间做出选择 | 官方博客（Anthropic，针对 Claude） |
| 决策模型 Choice<br><small>[System One 决策原语](https://docs.system-one.dev/zh-CN/docs/primitives) · 2026-10-11</small> | 从候选中选一个（可加审核选项），返回概率和 confidence<br><small>[System One 决策原语](https://docs.system-one.dev/zh-CN/docs/primitives) · 2026-10-11</small> | 不检索、不执行工具、不生成参数；结果不是授权<br><small>[System One Agent 接入文档](https://docs.system-one.dev/zh-CN/docs/agent-integration) · 2026-10-11</small> | 官方文档 |
| 主模型或代码生成参数并执行<br><small>[System One Agent 接入文档](https://docs.system-one.dev/zh-CN/docs/agent-integration) · 2026-10-11</small> | 生成参数，在权限检查后调用工具 | 选错工具的问题要在上一层解决；参数有效率需要单独度量 | 官方文档 + 本文建议 |
| System One 托管决策 API：system-one.dev 提供 Jev（`typesafe/jev`）、Clef Flash（`cloudflare/clef-flash`）和 Clef（`cloudflare/clef`）模型服务<br><small>[System One 托管 API 页](https://system-one.dev/zh-CN/api) · 2026-10-11</small> | 用 Choice 在候选中选择，每个问题最多 255 个选项<br><small>[System One 决策原语](https://docs.system-one.dev/zh-CN/docs/primitives) · 2026-10-11</small> | 不检索、不执行工具、不生成参数；结果不是授权<br><small>[System One Agent 接入文档](https://docs.system-one.dev/zh-CN/docs/agent-integration) · 2026-10-11</small> | 官方文档 |
## 常见问题

### 为什么工具越多，Agent 越容易选错？

候选越多，描述相近的工具越多，上下文也越拥挤。Anthropic 的文档说，工具数超过 30 到 50 个后，Claude 选对工具的能力会下降；一个常见的多服务器配置，光工具定义就可能占用约 5.5 万个 token（厂商针对自家模型的说法）。

### 检索、懒加载和决策模型分别解决哪一段？

检索和懒加载解决“从大量工具中找出少数候选”，同时减少上下文占用；决策模型的 Choice 解决“从候选中选一个，或判断都不合适”，并给出概率；主模型或代码负责生成参数并执行。决策模型不能代替检索：正确的工具不在候选里，它就选不到。

### 怎么用 Choice 从候选工具中选择？低置信时怎么办？

把候选工具名作为选项键，把“何时使用”写成选项描述，再加一个“都不合适”或“交回主模型”的选项。置信度高时交给应用的权限检查和调度器；置信度低或缺失时，交回主模型、询问用户或转人工。阈值要在自己的评估集上按风险分别设定。

### System One 的 Choice 最多支持多少个选项？

System One 文档规定，每个 Choice 问题最多 255 个选项。但工具路由仍建议从小型白名单开始，先把候选缩小，再交给 Choice。

### 选错工具和参数错怎么分别度量？

在带标注的评估集上，分别统计选择准确率（选中的工具是否符合预期）、漏判率（该转人工时有没有转）和参数有效率（选对工具后参数能否通过校验、调用成功）。选择准确率衡量的是决策步骤，参数有效率衡量的是主模型或代码。

### 工具路由和模型路由有什么区别？

工具路由决定这一步调用哪个工具，模型路由决定把请求交给哪个模型。两者都是从预设选项中做选择，都可以用决策模型实现；但工具路由看的是选择准确率和参数有效率，模型路由看的是质量、成本和延迟。