Ask about one rule at a time
A generic "is this toxic?" question mixes harassment, spam, self-harm and copyright into one label, and gives an operator nothing to act on. Name the rule — "does this attack a protected group?" — and send the rule text with the post.
The demo on this site shows the difference: a joke about Monday meetings stays up, while abuse aimed at a group is flagged. Same length, same tone, different rule outcome.
Question shape
tsimport { booleanQuestion } from '@system-one-ai/core';
export const policyQuestions = {
groupAttack: booleanQuestion(
'Does this post attack, dehumanize, or demean a protected group?',
),
spamLink: booleanQuestion(
'Is this post primarily an unsolicited link or product pitch?',
),
};
State carries the post text and the policy paragraph the reviewer would cite. Without the rule text, the judgement drifts toward whatever the model absorbed in training.
Illustrative code, not tested. The band values are placeholders.
tsexport async function moderate(client, post: { text: string; policy: string; context?: string }) {
const result = await client.evaluate({ state: post, questions: policyQuestions });
const attack = result.answers.groupAttack.probability;
const spam = result.answers.spamLink.probability;
if (attack >= 0.85) return 'block';
if (attack >= 0.45 || spam >= 0.7) return 'review';
return 'publish';
}
Bands, not labels
Each band maps to a different cost of being wrong:
- Block only above a high probability, because a wrong block is visible to the author and hard to appeal.
- Review in the middle: a queue with the probability attached, so reviewers prioritise.
- Publish below the floor, and keep the number for later calibration.
What it will not do
- Write your policy. It applies the text you send; ambiguous rules produce ambiguous verdicts.
- Read images or video. Jev reads text and JSON. Moderate the transcript or caption, or use a model that reads images.
- Handle appeals. A person decides those, with the probability as context, not as evidence.
For the general pattern of gating actions on a probability, see agent guardrails.
Cost
Every 4KB is one block, and anything smaller still counts as one block. Cost = blocks × questions × the model rate. Two questions on a short post is two credits on Jev.
New accounts start with 100 credits, and credits never expire. See pricing.
Run the moderation sample (sign-in required) to see a flagged and a passing post.