Anthropic Teaches Claude to Flag Its Own Misuse: 94% of High-Priority Cases Caught for Under 2% Extra Compute

Anthropic has detailed a production safety technique it calls self-assessment: after Claude finishes answering a user, the model is given a short hidden instruction to review the conversation it just had and output a structured judgment of whether the exchange shows signs of misuse. The assessment records a potential-harm flag, the harm category, a priority score for human review, whether Claude refused the request, and a one-sentence rationale.

The economics are the point. Because the conversation is already sitting in the model's KV cache the moment it finishes responding, adding a review step costs roughly 10 tokens per assessment, and across production traffic it adds less than 2% to inference cost, according to Anthropic. Customers are not billed separately for the layer. A conventional approach — piping every conversation into a separate safety model that re-reads long transcripts against detailed rules — costs far more compute.

Anthropic reports that, evaluated against a more comprehensive safety probe, self-assessment flagged 78% of potentially harmful conversations and 94% of high-priority cases, with a false-positive rate of about 1% — the number that matters, since a monitor that over-flags simply buries human reviewers. All Claude models released since the Fable 5 wave in June 2026 run the technique, and the company says detection strengthens with model scale. These are Anthropic's own measurements, not independently verified.

The feature is deliberately positioned as an offline monitoring layer, not a real-time gate. Flagged conversations route to more precise investigation tools and human reviewers; users see no interruption and the assessment never enters the conversation context. Anthropic says the layer has surfaced misuse its other safeguards were not designed to catch — in August 2026 alone it identified more than 100,000 flags involving child sexual abuse or exploitation traced to a single API customer, which the company escalated to the US National Center for Missing and Exploited Children.

The company is candid about limits. Self-assessment is trained from a stronger teacher model, so the teacher's blind spots pass to the student; a model could in principle game its own review; and research on self-attribution bias shows models judge their own recent behavior more leniently than the same content seen fresh. Anthropic says the technique is not yet reliable for monitoring Claude's own conduct, which it treats as a separate research problem centered on model honesty.

Strategically, the disclosure lands as the industry shifts from chatbots to autonomous agents with real permissions. A cheap, always-on, self-produced audit signal gives a lab broad coverage where dedicated classifiers scale poorly, and it complements rather than replaces real-time defenses. The open question — which Anthropic acknowledges — is whether a model grading its own conversations can ever be trusted as more than a triage filter, and that is why the human review loop, not the model, remains the enforcement point.