Too Many Tools? Route Agent Tool Calls with a Choice Question (TypeScript)

Choice is the step after retrieval, not a replacement for it. Background: What is a decision model? · Switching providers: Jev alternatives.

Try it online (sign-in required): NPC next action

Why more tools means more wrong picks

Connect a few MCP servers and tool schemas quickly fill the context window. Anthropic's tool search docs describe two problems:

• Context bloat: a common multi-server setup (GitHub, Slack, Sentry, Grafana, Splunk) can use about 55K tokens on tool definitions before Claude starts working.
• Lower selection accuracy: Claude's ability to pick the right tool degrades once there are more than 30–50 tools.

That's Anthropic describing its own models; other models may degrade at different points. The cause is intuitive: more candidates means more look-alike tools (several "search" tools, several that can "send a message") and more irrelevant schemas to distract the model.

Three layers: retrieve → select → act

Most current fixes target the first layer: retrieval, lazy loading and rewriting tool descriptions. Put them next to the other two steps and tool routing has three layers:

1. Retrieve: narrow the candidates. Keyword, BM25 or vector retrieval, or a model-invoked "tool search", cuts hundreds of tools down to a few. Anthropic's tool search returns up to 5 tools by default and recommends keeping the 3–5 most-used tools always loaded and deferring the rest.
2. Select: pick one. A bounded choice: one of the candidates, or "none fits". This is where a Choice question fits.
3. Act: generate arguments and call. Your app's permission check plus the main LLM (or code) build the arguments and invoke the tool. System One doesn't call tools or generate arguments.

If retrieval fails, nothing downstream can recover: if the right tool isn't in the candidate set, no selector can pick it. A third-party head-to-head (Stacklok's blog, vendor-run) explains tool-selection accuracy gaps the same way. So a decision model doesn't replace retrieval; it handles the step after it.

The dev team's diagram (examples/decision-articles/04-three-tier-architecture.mmd; a rendered 04-three-tier-architecture.svg sits in the same folder). Retrieval belongs to the app, Choice only names a tool, and the confidence bands are app thresholds, not hosted defaults.

mermaid%% Article 4 — three-tier tool routing after Choice.
%% Retrieve is application-owned. Choice only names a tool.
%% Confidence bands are application thresholds, not a hosted default.
flowchart TB
  subgraph retrieve [Layer 1 - Retrieve]
    Message[User message] --> Catalog[Allowlisted catalog]
    Catalog --> Candidates[Candidate tools]
  end

  subgraph select [Layer 2 - Select]
    Candidates --> Choice[System One Choice]
    Choice --> Band{Confidence band}
  end

  subgraph act [Layer 3 - Act]
    Band -->|high| Execute[Propose execute tool]
    Band -->|medium| Confirm[Ask user to confirm]
    Band -->|low or missing| Fallback[Main application LLM]
    Confirm -->|approved| Execute
    Confirm -->|declined| Stop[Do not execute]
  end

Using Choice for tool selection

Choice picks one named option. The System One primitives docs say a native Choice answer includes the chosen option (choice), a probability distribution keyed by option name (probabilities), and a separately reported confidence statistic.

Option = tool name + one "when to use" line

The option key is the tool name and the value says when to pick it. The System One agent integration docs say tool descriptions should explain when to choose the option, must not contain credentials, and can't grant permissions. Keep options clearly distinct from each other.

Always leave a "none of these" exit

The primitives docs recommend adding a review option when forcing a category would mislead, and the integration docs suggest an explicit think or review option where it fits, so the model doesn't have to force a tool. The examples below use defer_to_llm (hand back to the main LLM); the allowlist also has two non-tool options, ask_user and human_review.

Choice allows 255 options, so why narrow first?

System One allows up to 255 options per Choice question. For classification, the Jev docs even recommend giving the full list rather than a shortlist, since each extra option costs only a few tokens.

Tool routing is different. The System One integration docs say to start from a small allowlist rather than every tool the agent can reach, and never let model-generated names, URLs or shell commands bypass it. Two reasons: tool calls have real consequences, so the candidate set is a permission boundary; and a tool's "when to use" text is usually much longer than a class label. Rule of thumb: with few candidates, put them straight into Choice; with many tools, long descriptions or mixed permissions, retrieve first, then Choice. Which is more accurate for you depends on your own eval set; we haven't benchmarked it.

Confidence thresholds and fallbacks

After the Choice result arrives, your app decides the next step by confidence. The Jev confidence docs suggest three bands as a starting point:

• High: act automatically. In tool routing, hand off to the app's permission check and dispatcher.
• Medium: proceed carefully: ask the user to confirm, flag for review, or gather more information.
• Low: don't act. Send to a person, ask the user to clarify, or fall back to another system such as the main LLM.

Key points:

• Confidence isn't "probability of being right". The System One primitives docs say confidence doesn't necessarily equal the chosen option's probability, and neither is a universal action threshold. The Jev docs add that Jev's Choice confidence is computed from the distribution: 1 when all probability is on one option, 0 when spread evenly. They also suggest looking at the top probability and the ratio between the first and second options.
• Thresholds scale with risk. Read-only lookups can use a lower bar. Payments, permission changes and outbound messages need higher confidence or human confirmation regardless; the integration docs require policy-mandated confirmation for these.
• Missing means unresolved. A missing answer or failed request isn't an automatic yes, no or lowest score. The examples never act when confidence is missing.
• A model result is not authorization. Choosing a tool doesn't mean this user may run it. Check identity, permissions and approvals outside the model.
• Where thresholds come from: your own eval cases and consequence analysis, not a universal percentage. Re-evaluate after changing the model, instructions, options or descriptions.

The three-band version

The dev team's examples/decision-articles/04-three-tier-routing.ts imports the allowlist, retrieval function and DEFER_TO_LLM from the main example (04-agent-tool-routing.ts, shown further down). At or above HIGH_EXECUTE_CONFIDENCE (0.75) it returns "execute". Between MEDIUM_CONFIRM_CONFIDENCE (0.45) and the high bar it calls your confirm callback. Below that, when confidence is missing, or when the model picks defer_to_llm, it falls back to your main LLM. Both thresholds are examples; tune them on labelled cases and action risk.

ts/**
 * Article 4 — Three-tier tool routing after Choice.
 *
 * TypeSafe's confidence guidance is a three-band starting point: high →
 * execute, medium → confirm with the user, low → do not execute. System One
 * only returns Choice + optional confidence. Thresholds, the confirm UI, and
 * the main-LLM fallback are application-owned. Missing confidence stays
 * unknown and is treated as "do not execute".
 *
 * This file does not generate tool arguments and does not call tools.
 */
import { choice } from '@system-one-ai/core';
import type { ChoiceAnswer, EvaluationClient } from '@system-one-ai/core';
import {
  DEFER_TO_LLM,
  TOOL_CATALOG,
  createHostedClient,
  retrieveCandidateTools,
} from './04-agent-tool-routing.ts';

/** Application-owned. Tune on labeled cases and action risk; not a hosted default. */
export const HIGH_EXECUTE_CONFIDENCE = 0.75;
export const MEDIUM_CONFIRM_CONFIDENCE = 0.45;

export type RouteTier = 'execute' | 'confirm' | 'fallback';

export function buildToolChoiceCriteria(
  tools: readonly { id: string; whenToUse: string }[],
) {
  return Object.fromEntries([
    ...tools.map(tool => [tool.id, tool.whenToUse]),
    [DEFER_TO_LLM, 'None of the allowlisted tools fits; continue with the main application LLM'],
  ]);
}

export function classifyRouteTier(answer: ChoiceAnswer<string>): RouteTier {
  if (answer.choice === DEFER_TO_LLM) return 'fallback';
  if (answer.confidence === undefined) return 'fallback';
  if (answer.confidence >= HIGH_EXECUTE_CONFIDENCE) return 'execute';
  if (answer.confidence >= MEDIUM_CONFIRM_CONFIDENCE) return 'confirm';
  return 'fallback';
}

export interface ConfirmToolInput {
  message: string;
  proposedTool: string;
  confidence: number;
}

/**
 * @param askUserToConfirm Your confirm UI or approval queue. Not a System One method.
 * @param completeWithApplicationLlm Your agent's existing completion path.
 */
export async function routeToolThreeTier(
  client: EvaluationClient,
  message: string,
  callbacks: {
    askUserToConfirm: (input: ConfirmToolInput) => Promise<boolean>;
    completeWithApplicationLlm: (prompt: string) => Promise<string>;
  },
) {
  const candidates = retrieveCandidateTools(message, TOOL_CATALOG);
  const result = await client.evaluate({
    state: { message, retrievedTools: candidates.map(tool => tool.id) },
    questions: {
      tool: choice('Choose the next allowed tool. Do not execute it.', buildToolChoiceCriteria(candidates)),
    },
  });
  const answer = result.answers.tool;
  const tier = classifyRouteTier(answer);

  if (tier === 'execute') {
    return {
      action: 'execute_tool' as const,
      tier,
      proposedTool: answer.choice,
      confidence: answer.confidence,
    };
  }

  if (tier === 'confirm') {
    const approved = await callbacks.askUserToConfirm({
      message,
      proposedTool: answer.choice,
      confidence: answer.confidence as number,
    });
    return approved
      ? {
          action: 'execute_tool' as const,
          tier,
          proposedTool: answer.choice,
          confidence: answer.confidence,
          confirmedByUser: true as const,
        }
      : {
          action: 'user_declined' as const,
          tier,
          proposedTool: answer.choice,
          confidence: answer.confidence,
        };
  }

  const fallback = await callbacks.completeWithApplicationLlm(message);
  return {
    action: 'fallback_llm' as const,
    tier,
    proposedTool: answer.choice,
    confidence: answer.confidence,
    fallback,
  };
}

async function main() {
  const message = process.env.EXAMPLE_MESSAGE ?? 'Can you explain how credits are calculated?';
  const decision = await routeToolThreeTier(createHostedClient(), message, {
    askUserToConfirm: async () => {
      throw new Error('No System One confirm UI. Wire your approval callback here.');
    },
    completeWithApplicationLlm: async () => {
      throw new Error('No System One LLM-completion endpoint. Wire your existing model call here.');
    },
  });
  console.log(JSON.stringify(decision, null, 2));
}

if (process.argv[1] && import.meta.url.endsWith(process.argv[1].split(/[\\/]/).pop() ?? '')) {
  main().catch((error: unknown) => {
    console.error(error instanceof Error ? error.message : error);
    process.exitCode = 1;
  });
}

Measure selection accuracy and argument validity separately

Tool calls fail in two distinct ways: the wrong tool, or the right tool with wrong arguments. In the three-layer model these are layer-2 and layer-3 failures, so count them separately:

• Selection accuracy: on a labelled eval set, how often Choice matches the expected tool. Also track missed escalations (should have gone to a person but didn't). The System One docs recommend counting wrong-tool selections and missed reviews separately, and including ambiguous requests and instructions that try to force a different tool.
• Retrieval recall: how often the right tool is in the candidate set. This caps selection accuracy.
• Argument validity: given the right tool, how often arguments pass validation and the call succeeds. That's the main LLM's or your code's job, not the decision model's.
• Fallback rate: how often requests go back to the main LLM, the user or a person. This drives cost and UX.

The docs also note that an HTTP 200 and a high probability don't prove the routing is safe. Run in suggestion-only mode first, then decide by policy which read-only actions may run automatically.

Generating arguments after routing

examples/decision-articles/04-generate-tool-params.ts calls your own argument-generation callback only after the three-band router says "execute", then checks required fields with paramsAreValid(). Choice only names the tool; System One doesn't generate, validate or call anything. TOOL_PARAM_REQUIRED is for illustration.

ts/**
 * Article 4 — Generate tool-call parameters after routing.
 *
 * Choice only names the tool. System One does not emit arguments, validate
 * them, or invoke the tool. Parameter generation stays in the application
 * (deterministic code or your main LLM). Measure parameter validity separately
 * from routing accuracy.
 */
import type { EvaluationClient } from '@system-one-ai/core';
import { DEFER_TO_LLM, createHostedClient } from './04-agent-tool-routing.ts';
import { routeToolThreeTier } from './04-three-tier-routing.ts';

export interface GeneratedToolParams {
  readonly [key: string]: string;
}

export const TOOL_PARAM_REQUIRED: Readonly<Record<string, readonly string[]>> = {
  search_docs: ['query'],
  ask_user: ['question'],
  human_review: ['reason'],
};

export function paramsAreValid(toolId: string, params: unknown): boolean {
  const required = TOOL_PARAM_REQUIRED[toolId];
  if (!required) return false;
  if (typeof params !== 'object' || params === null || Array.isArray(params)) return false;
  const record = params as Record<string, unknown>;
  return required.every(key => typeof record[key] === 'string' && record[key].trim().length > 0);
}

export interface GenerateParamsInput {
  message: string;
  toolId: string;
}

/**
 * @param generateParams Application-owned. Not a System One method.
 *        The decision model has already picked `toolId` and must not invent args.
 */
export async function routeThenGenerateParams(
  client: EvaluationClient,
  message: string,
  generateParams: (input: GenerateParamsInput) => Promise<unknown>,
  callbacks: {
    askUserToConfirm: (input: { message: string; proposedTool: string; confidence: number }) => Promise<boolean>;
    completeWithApplicationLlm: (prompt: string) => Promise<string>;
  },
) {
  const routed = await routeToolThreeTier(client, message, callbacks);
  if (routed.action !== 'execute_tool' || routed.proposedTool === DEFER_TO_LLM) {
    return { ...routed, params: undefined, paramsValid: false as const };
  }
  const params = await generateParams({ message, toolId: routed.proposedTool });
  return {
    ...routed,
    params,
    paramsValid: paramsAreValid(routed.proposedTool, params),
  };
}

async function main() {
  const message = process.env.EXAMPLE_MESSAGE ?? 'Can you explain how credits are calculated?';
  const decision = await routeThenGenerateParams(
    createHostedClient(),
    message,
    async ({ message: userMessage, toolId }) => {
      if (toolId === 'search_docs') return { query: userMessage };
      throw new Error(`No application parameter builder wired for ${toolId}.`);
    },
    {
      askUserToConfirm: async () => true,
      completeWithApplicationLlm: async () => {
        throw new Error('No System One LLM-completion endpoint. Wire your existing model call here.');
      },
    },
  );
  console.log(JSON.stringify(decision, null, 2));
}

if (process.argv[1] && import.meta.url.endsWith(process.argv[1].split(/[\\/]/).pop() ?? '')) {
  main().catch((error: unknown) => {
    console.error(error instanceof Error ? error.message : error);
    process.exitCode = 1;
  });
}

Offline eval: recall, selection accuracy, missed reviews, argument validity

examples/decision-articles/04-routing-eval-harness.ts computes four metrics on a labelled set: retrieval recall, Choice accuracy, the missed-review rate, and argument validity among correctly routed rows. System One has no eval-dataset or judge API; this is all application code. The four ROUTE_EVAL_CASES are demo data; replace them before tuning thresholds.

ts/**
 * Article 4 — Offline routing-accuracy harness on a labeled set.
 *
 * System One has no eval-dataset or judge API. This file scores:
 *   1. retrieval recall — expected tool present in the candidate set
 *   2. choice accuracy — Choice equals the labeled tool
 *   3. missed-review rate — labeled human_review but Choice picked something else
 * Parameter validity is a separate column, computed only on correctly routed
 * rows, because argument generation is not a decision-model job.
 */
import { choice } from '@system-one-ai/core';
import type { EvaluationClient } from '@system-one-ai/core';
import {
  DEFER_TO_LLM,
  TOOL_CATALOG,
  createHostedClient,
  retrieveCandidateTools,
} from './04-agent-tool-routing.ts';
import { paramsAreValid } from './04-generate-tool-params.ts';
import { buildToolChoiceCriteria } from './04-three-tier-routing.ts';

export interface LabeledRouteCase {
  id: string;
  message: string;
  expectedTool: string;
  /** When set, used only after a correct Choice to score argument generation. */
  expectedParams?: Readonly<Record<string, string>>;
}

/** Small labeled set for the article. Replace with your own cases before tuning thresholds. */
export const ROUTE_EVAL_CASES: readonly LabeledRouteCase[] = [
  { id: 'credits', message: 'How are credits calculated?', expectedTool: 'search_docs' },
  { id: 'vague', message: 'It broke and does not work', expectedTool: 'ask_user' },
  { id: 'refund', message: 'I was charged twice and want a refund', expectedTool: 'human_review' },
  { id: 'unrelated', message: 'Write a haiku about rain', expectedTool: DEFER_TO_LLM },
];

export interface RouteEvalRow {
  id: string;
  expectedTool: string;
  predictedTool: string | undefined;
  retrieved: readonly string[];
  retrievedExpected: boolean;
  correctChoice: boolean;
  missedReview: boolean;
  paramsValid: boolean | undefined;
}

export function evaluateRetrieval(cases: readonly LabeledRouteCase[] = ROUTE_EVAL_CASES) {
  return cases.map((labeled) => {
    const retrieved = retrieveCandidateTools(labeled.message, TOOL_CATALOG).map(tool => tool.id);
    const retrievedExpected =
      labeled.expectedTool === DEFER_TO_LLM || retrieved.includes(labeled.expectedTool);
    return { id: labeled.id, retrieved, retrievedExpected };
  });
}

export async function evaluateRoutingAccuracy(
  client: EvaluationClient,
  cases: readonly LabeledRouteCase[] = ROUTE_EVAL_CASES,
  generateParams?: (input: { message: string; toolId: string }) => Promise<unknown>,
) {
  const rows: RouteEvalRow[] = [];
  for (const labeled of cases) {
    const candidates = retrieveCandidateTools(labeled.message, TOOL_CATALOG);
    const retrieved = candidates.map(tool => tool.id);
    const retrievedExpected =
      labeled.expectedTool === DEFER_TO_LLM || retrieved.includes(labeled.expectedTool);
    const result = await client.evaluate({
      state: { message: labeled.message, retrievedTools: retrieved },
      questions: {
        tool: choice('Choose the next allowed tool. Do not execute it.', buildToolChoiceCriteria(candidates)),
      },
    });
    const predictedTool = result.answers.tool.choice;
    const correctChoice = predictedTool === labeled.expectedTool;
    const missedReview = labeled.expectedTool === 'human_review' && predictedTool !== 'human_review';

    let paramsValid: boolean | undefined;
    if (correctChoice && labeled.expectedParams && generateParams) {
      const params = await generateParams({ message: labeled.message, toolId: predictedTool });
      paramsValid = paramsAreValid(predictedTool, params);
    }

    rows.push({
      id: labeled.id,
      expectedTool: labeled.expectedTool,
      predictedTool,
      retrieved,
      retrievedExpected,
      correctChoice,
      missedReview,
      paramsValid,
    });
  }
  return rows;
}

export function summarizeRouteEval(rows: readonly RouteEvalRow[]) {
  const reviewCases = rows.filter(row => row.expectedTool === 'human_review');
  const correctWithParams = rows.filter(row => row.correctChoice && row.paramsValid !== undefined);
  return {
    n: rows.length,
    retrievalRecall: rows.length === 0 ? 0 : rows.filter(row => row.retrievedExpected).length / rows.length,
    choiceAccuracy: rows.length === 0 ? 0 : rows.filter(row => row.correctChoice).length / rows.length,
    missedReviewRate: reviewCases.length === 0
      ? 0
      : reviewCases.filter(row => row.missedReview).length / reviewCases.length,
    paramValidityAmongCorrect: correctWithParams.length === 0
      ? undefined
      : correctWithParams.filter(row => row.paramsValid).length / correctWithParams.length,
  };
}

async function main() {
  const retrieval = evaluateRetrieval();
  console.log(JSON.stringify({ retrieval, retrievalOnly: true }, null, 2));
  if (!process.env.SYSTEM_ONE_API_KEY) return;
  const rows = await evaluateRoutingAccuracy(createHostedClient());
  console.log(JSON.stringify({ rows, summary: summarizeRouteEval(rows) }, null, 2));
}

if (process.argv[1] && import.meta.url.endsWith(process.argv[1].split(/[\\/]/).pop() ?? '')) {
  main().catch((error: unknown) => {
    console.error(error instanceof Error ? error.message : error);
    process.exitCode = 1;
  });
}

Tool routing vs model routing

Both are a choice among predefined options and both can use a decision model, but the options and metrics differ:

• Tool routing decides which tool (if any) to call next. Metrics: selection accuracy, missed escalations, argument validity.
• Model routing decides which model handles the request. Microsoft's Microsoft-Decision-1 launch blog describes it as selecting the best model for quality, cost and latency requirements (vendor statement). Metrics: quality, cost, latency.

Microsoft lists routing and classification among Microsoft-Decision-1's uses (vendor statement). For choosing and switching between decision models, see Jev alternatives. For a decision model as an eval first pass, see An LLM-as-a-judge alternative. For difficulty-based routing between small and large models, see LLM model routing.

TypeScript: retrieve candidates → Choice → fall back on low confidence

The dev team's main example (examples/decision-articles/04-agent-tool-routing.ts) retrieves candidates from a static allowlist by keyword, asks a Choice with a defer_to_llm option, and proposes a tool only when confidence exists and meets the threshold; otherwise it falls back to your main LLM.

ts/**
 * Article 4 — Agent tool routing: retrieve candidates, Choice, fall back on low confidence.
 *
 * System One can pick one named option from the set you send. It does not
 * retrieve tools, execute them, or run a general LLM completion. Retrieval
 * and the "main LLM" fallback are application functions in this file.
 *
 * Confidence is an optional provider statistic, not necessarily P(selected
 * tool) and not a universal threshold. Missing confidence stays unknown —
 * this example treats unknown as "do not execute".
 */
import { createSystemOne, choice } from '@system-one-ai/core';
import type { ChoiceAnswer, EvaluationClient } from '@system-one-ai/core';
import { systemOneAdapter } from '@system-one-ai/adapter-system-one';
import { createFetchTransport } from '@system-one-ai/transport-fetch';

export interface ToolSpec {
  id: string;
  whenToUse: string;
  keywords: readonly string[];
}

/** Static allowlist. There is no System One tool-catalog endpoint. */
export const TOOL_CATALOG: readonly ToolSpec[] = [
  { id: 'search_docs', whenToUse: 'A product or usage question can be answered from documentation', keywords: ['credit', 'docs', 'how', 'price'] },
  { id: 'ask_user', whenToUse: 'Essential details are missing; ask before proceeding', keywords: ['it broke', 'does not work', 'help'] },
  { id: 'human_review', whenToUse: 'A payment dispute, refund, or policy exception needs a person', keywords: ['refund', 'charged', 'dispute'] },
];

export const DEFER_TO_LLM = 'defer_to_llm';
export const MIN_EXECUTE_CONFIDENCE = 0.55;

export function retrieveCandidateTools(message: string, catalog: readonly ToolSpec[] = TOOL_CATALOG) {
  const text = message.toLowerCase();
  const hits = catalog.filter(tool => tool.keywords.some(keyword => text.includes(keyword)));
  return hits.length > 0 ? hits : catalog.filter(tool => tool.id === 'ask_user' || tool.id === 'human_review');
}

export function shouldExecuteChosenTool(answer: ChoiceAnswer<string>) {
  if (answer.choice === DEFER_TO_LLM) return false;
  if (answer.confidence === undefined) return false;
  return answer.confidence >= MIN_EXECUTE_CONFIDENCE;
}

export function createHostedClient() {
  const apiKey = process.env.SYSTEM_ONE_API_KEY;
  const baseURL = process.env.SYSTEM_ONE_BASE_URL;
  const model = process.env.SYSTEM_ONE_MODEL;
  if (!apiKey || !baseURL || !model) {
    throw new Error('Set SYSTEM_ONE_API_KEY, SYSTEM_ONE_BASE_URL (…/v1), and SYSTEM_ONE_MODEL from GET /v1/models.');
  }
  return createSystemOne({
    apiKey, baseURL, model, adapter: systemOneAdapter,
    transport: createFetchTransport(), timeoutMs: 30_000, maxRetries: 0,
  });
}

/**
 * @param completeWithApplicationLlm Your agent's existing completion path.
 *        System One has no chat/completions URL for this fallback.
 */
export async function routeTool(
  client: EvaluationClient,
  message: string,
  completeWithApplicationLlm: (prompt: string) => Promise<string>,
) {
  const candidates = retrieveCandidateTools(message);
  const criteria = Object.fromEntries([
    ...candidates.map(tool => [tool.id, tool.whenToUse]),
    [DEFER_TO_LLM, 'None of the allowlisted tools fits; continue with the main application LLM'],
  ]);
  const result = await client.evaluate({
    state: { message, retrievedTools: candidates.map(tool => tool.id) },
    questions: {
      tool: choice('Choose the next allowed tool. Do not execute it.', criteria),
    },
  });
  const answer = result.answers.tool;
  if (!shouldExecuteChosenTool(answer)) {
    const fallback = await completeWithApplicationLlm(message);
    return {
      action: 'fallback_llm' as const,
      proposedTool: answer.choice,
      confidence: answer.confidence,
      fallback,
    };
  }
  return {
    action: 'propose_tool' as const,
    proposedTool: answer.choice,
    confidence: answer.confidence,
  };
}

async function main() {
  const message = process.env.EXAMPLE_MESSAGE ?? 'Can you explain how credits are calculated?';
  const decision = await routeTool(createHostedClient(), message, async () => {
    throw new Error('No System One LLM-completion endpoint. Wire your existing model call here.');
  });
  console.log(JSON.stringify(decision, null, 2));
}

if (process.argv[1] && import.meta.url.endsWith(process.argv[1].split(/[\\/]/).pop() ?? '')) {
  main().catch((error: unknown) => {
    console.error(error instanceof Error ? error.message : error);
    process.exitCode = 1;
  });
}

Notes (from the example's code and comments):

• Retrieval and fallback are your functions. System One doesn't retrieve or execute tools, has no general LLM completion endpoint and has no tool-catalog endpoint; TOOL_CATALOG is a static allowlist in code. The keyword matcher is a placeholder; use BM25 or vector retrieval in production.
• Propose, don't execute. When confident, the example returns action: 'propose_tool' and leaves execution to your permission check and dispatcher.
• The threshold is an example. MIN_EXECUTE_CONFIDENCE = 0.55; re-pick it on your eval set. Missing (undefined) confidence means "don't execute".
• Hosted model: take SYSTEM_ONE_MODEL from GET /v1/models. A full model ID selects a model but can't pin a version.

The three-band and measurement sections above add the confirm tier, argument generation and offline eval. The next two examples show how to wire routing into the Vercel AI SDK and into an MCP client you already run.

With the Vercel AI SDK

examples/decision-articles/04-vercel-ai-sdk-tools.ts defines the three allowlisted tools with the AI SDK's tool(), routes with the three-band router, then calls generateText with activeTools and toolChoice to restrict the model to that one tool, so the language model bound to the AI SDK fills in the arguments. Per its comments, @system-one-ai/adapter-vercel is an Evaluation adapter, not an AI SDK tools adapter, and isn't a dependency of this repo, so it isn't imported. The model passed to generateText is your own model provider, not System One. The example targets ai@6.0.82.

ts/**
 * Article 4 — Vercel AI SDK tools wrapping System One routing.
 *
 * `@system-one-ai/adapter-vercel` is a published Vercel *Evaluation* adapter
 * (not an AI SDK tools adapter) and is not a dependency of this repo. This
 * file uses the already-installed `ai` package: `tool()`, `generateText`,
 * `activeTools`, and `toolChoice: { type: 'tool', toolName }`.
 *
 * Choice still only picks the tool. The language model bound to `generateText`
 * fills `inputSchema`. System One does not generate arguments.
 */
import { generateText, tool } from 'ai';
import type { LanguageModel, ToolSet } from 'ai';
import { z } from 'zod';
import type { EvaluationClient } from '@system-one-ai/core';
import { createHostedClient } from './04-agent-tool-routing.ts';
import { routeToolThreeTier } from './04-three-tier-routing.ts';

export const searchDocsTool = tool({
  description: 'A product or usage question can be answered from documentation',
  inputSchema: z.object({
    query: z.string().describe('Documentation search query'),
  }),
  execute: async ({ query }) => ({ source: 'application-docs-search', query }),
});

export const askUserTool = tool({
  description: 'Essential details are missing; ask before proceeding',
  inputSchema: z.object({
    question: z.string().describe('Clarifying question for the user'),
  }),
  execute: async ({ question }) => ({ source: 'application-ask-user', question }),
});

export const humanReviewTool = tool({
  description: 'A payment dispute, refund, or policy exception needs a person',
  inputSchema: z.object({
    reason: z.string().describe('Why a person must review this request'),
  }),
  execute: async ({ reason }) => ({ source: 'application-human-review', reason }),
});

export const allowlistedAiTools = {
  search_docs: searchDocsTool,
  ask_user: askUserTool,
  human_review: humanReviewTool,
} satisfies ToolSet;

export type AllowlistedToolName = keyof typeof allowlistedAiTools;

export function routedGenerateTextOptions(
  proposedTool: AllowlistedToolName,
  message: string,
) {
  return {
    prompt: message,
    tools: allowlistedAiTools,
    activeTools: [proposedTool] as Array<AllowlistedToolName>,
    toolChoice: { type: 'tool' as const, toolName: proposedTool },
  };
}

/**
 * Route with System One, then let the AI SDK model fill that tool's arguments.
 *
 * @param model A Vercel AI SDK `LanguageModel` from your existing provider
 *        package (for example `@ai-sdk/openai`). Not a System One model.
 */
export async function routeThenGenerateText(
  client: EvaluationClient,
  model: LanguageModel,
  message: string,
  callbacks: {
    askUserToConfirm: (input: { message: string; proposedTool: string; confidence: number }) => Promise<boolean>;
    completeWithApplicationLlm: (prompt: string) => Promise<string>;
  },
) {
  const routed = await routeToolThreeTier(client, message, callbacks);
  if (routed.action !== 'execute_tool' || !(routed.proposedTool in allowlistedAiTools)) {
    return { routed, generation: undefined };
  }
  const proposedTool = routed.proposedTool as AllowlistedToolName;
  const generation = await generateText({
    model,
    ...routedGenerateTextOptions(proposedTool, message),
  });
  return { routed, generation };
}

async function main() {
  const message = process.env.EXAMPLE_MESSAGE ?? 'Can you explain how credits are calculated?';
  const routed = await routeToolThreeTier(createHostedClient(), message, {
    askUserToConfirm: async () => true,
    completeWithApplicationLlm: async () => {
      throw new Error('No System One LLM-completion endpoint. Wire your existing model call here.');
    },
  });
  if (routed.action === 'execute_tool' && routed.proposedTool in allowlistedAiTools) {
    console.log(JSON.stringify({
      routed,
      generateTextOptions: routedGenerateTextOptions(routed.proposedTool as AllowlistedToolName, message),
    }, null, 2));
    return;
  }
  console.log(JSON.stringify({ routed }, null, 2));
}

if (process.argv[1] && import.meta.url.endsWith(process.argv[1].split(/[\\/]/).pop() ?? '')) {
  main().catch((error: unknown) => {
    console.error(error instanceof Error ? error.message : error);
    process.exitCode = 1;
  });
}

With MCP tools

examples/decision-articles/04-mcp-tool-routing.ts uses only the client types from the official @modelcontextprotocol/sdk. It calls listTools() on an MCP client you've already connected, turns tool names and descriptions into Choice options (plus defer_to_llm), and returns the proposed tool and its inputSchema only when confidence meets the high threshold and the tool is actually in the list. Your app makes the real call with callTool(), with arguments from your code or main LLM. System One doesn't speak MCP and has no MCP integration; it doesn't retrieve or execute tools. This example sends every listed tool to Choice, so with large tool sets, retrieve first as described above.

ts/**
 * Article 4 — Route among tools advertised by an MCP client.
 *
 * Uses official `@modelcontextprotocol/sdk` client types only: `Client.listTools()`
 * and `Client.callTool({ name, arguments })`. System One does not speak MCP,
 * retrieve tools, or execute them. After Choice, the application calls MCP.
 */
import { choice } from '@system-one-ai/core';
import type { EvaluationClient } from '@system-one-ai/core';
import type { Client } from '@modelcontextprotocol/sdk/client';
import { DEFER_TO_LLM } from './04-agent-tool-routing.ts';
import { HIGH_EXECUTE_CONFIDENCE } from './04-three-tier-routing.ts';

export type McpToolLister = Pick<Client, 'listTools'>;
export type McpToolCaller = Pick<Client, 'callTool'>;
export type McpListedTool = Awaited<ReturnType<Client['listTools']>>['tools'][number];

export function mcpToolsToChoiceCriteria(tools: readonly McpListedTool[]) {
  return Object.fromEntries([
    ...tools.map(tool => [
      tool.name,
      tool.description ?? `MCP tool ${tool.name}. Use only when this tool is the best next action.`,
    ]),
    [DEFER_TO_LLM, 'None of the listed MCP tools fits; continue with the main application LLM'],
  ]);
}

export async function listMcpTools(mcp: McpToolLister) {
  const listed = await mcp.listTools();
  return listed.tools;
}

/**
 * List MCP tools, then let Choice pick one name. Does not call the tool.
 */
export async function routeMcpTool(
  client: EvaluationClient,
  mcp: McpToolLister,
  message: string,
) {
  const tools = await listMcpTools(mcp);
  if (tools.length === 0) {
    return {
      action: 'fallback_llm' as const,
      proposedTool: DEFER_TO_LLM,
      confidence: undefined,
      available: [] as string[],
    };
  }
  const result = await client.evaluate({
    state: { message, retrievedTools: tools.map(tool => tool.name) },
    questions: {
      tool: choice('Choose the next MCP tool. Do not execute it.', mcpToolsToChoiceCriteria(tools)),
    },
  });
  const answer = result.answers.tool;
  const available = tools.map(tool => tool.name);
  if (
    answer.choice === DEFER_TO_LLM
    || answer.confidence === undefined
    || answer.confidence < HIGH_EXECUTE_CONFIDENCE
    || !available.includes(answer.choice)
  ) {
    return {
      action: 'fallback_llm' as const,
      proposedTool: answer.choice,
      confidence: answer.confidence,
      available,
    };
  }
  return {
    action: 'propose_mcp_tool' as const,
    proposedTool: answer.choice,
    confidence: answer.confidence,
    available,
    inputSchema: tools.find(tool => tool.name === answer.choice)?.inputSchema,
  };
}

/**
 * Application-owned execution step. Arguments come from your code or main LLM,
 * not from System One.
 */
export async function callRoutedMcpTool(
  mcp: McpToolCaller,
  name: string,
  args: Record<string, unknown>,
) {
  return mcp.callTool({ name, arguments: args });
}

async function main() {
  throw new Error(
    'This file routes tools from an MCP Client you already connected. There is no System One MCP endpoint.',
  );
}

if (process.argv[1] && import.meta.url.endsWith(process.argv[1].split(/[\\/]/).pop() ?? '')) {
  main().catch((error: unknown) => {
    console.error(error instanceof Error ? error.message : error);
    process.exitCode = 1;
  });
}

For auth, idempotency and retries, see the agent integration docs.

These combine. Anthropic's figures are about Claude and don't describe other models. We haven't run comparable tests.

Hosted billing: every 4KB of request is one block, and anything under 4KB still counts as one; cost = blocks × questions × the model's price per block. New accounts get 100 free credits on sign-up. See pricing.

Related reading

• What is a decision model?
• AI agent stop conditions
• AI agent guardrails
• LLM model routing