Key facts
What counts as a false positive
Three different layers can decline a request, and only one of them is what the research literature means by overrefusal. The model can decline on its own, having been trained toward the behavior described in Claude’s Constitution. An input or output classifier running alongside the model can flag the request before or after generation. Or the surrounding product can block the action for plan, region, account, or policy reasons.
A classifier false positive is the middle case: a harmless request caught by a guardrail the model never consulted. The practical tell is portability. If the identical request succeeds on a different Claude model, or through a different surface, the boundary was probably not the model’s judgment about your request.
- Model-level refusal: trained behavior; responds to clarified context and legitimate framing.
- Classifier intervention: a separate system flags input or output; scope is set per model, not per prompt.
- Product enforcement: plan, region, account, or surface blocks the action before the model is reached.
The measured history
Anthropic has published more numbers on this than most labs. The table collects the figures that bear directly on false positives, with the date each was published. None of them is a current production overrefusal rate, and no source on this page claims to be one.
| Published | What was reported | Why it matters for false positives |
|---|---|---|
| Feb 2025 | A prototype Constitutional Classifiers system survived an estimated 3,000+ hours of human red teaming with no universal jailbreak found, but with "high overrefusal rates and compute overhead." | Robustness was reachable. The first version that reached it was not deployable, and Anthropic said so in the announcement. |
| Feb 2025 | The updated system cut synthetic jailbreak success from 86% to 4.4%, raised the refusal rate by 0.38% (not statistically significant across a random sample of 5,000 conversations), and added 23.7% compute cost. | This 0.38% is the only published false-positive figure for a Claude classifier system. It is specific to that system, that model, and that date. |
| Feb 2025 | In the public red-team demo, 339 jailbreakers ran more than 300,000 chats; the system held for five days, then one participant found what Anthropic judged a universal jailbreak. | A classifier tuned for low overrefusal is not a perfect filter. The tradeoff runs in both directions. |
| Feb 2025 | The demo’s own automated grader refused to grade roughly 1% of submissions overall, and about 10% on one question. | The measurement tooling produced false positives too. Worth remembering before treating any single classifier output as ground truth. |
| Jul 2026 | Opus 5 cyber classifiers are expected to intervene around 85% less often than those on Fable 5. They permit finding vulnerabilities in source code, while blocking binary-based scanning, penetration testing, and exploit generation. | Scope moved toward the risky act rather than the topic, which is the single largest documented reduction in security-adjacent false positives. |
| Jul 2026 | Flagged Opus 5 requests fall back to Opus 4.8 by default in Claude.ai, Claude Code, and Claude Cowork; fallbacks are optional on the API. | A false positive increasingly looks like a quality drop rather than a refusal. See the next section. |
| Aug 2026 | Researchers in biology and chemistry remain limited to Opus-class models, and Fable models continue to block professional biology and drug-development queries. | A refusal can be a property of the model you were routed to rather than anything about your request. |
Fallbacks changed what a false positive looks like
Until 2026, a flagged request was a visible event. You got a refusal, and you knew a boundary had been hit. The Opus 5 launch introduced automatic fallbacks, described in the announcement as a way for requests to "always route to the best available model by default rather than being blocked."
For completion rates this is an improvement. For observability it is a regression, and it is worth naming plainly: a user who receives a quietly weaker answer from a fallback model cannot distinguish a classifier false positive from an ordinary bad response. The failure that used to announce itself now looks like the model having an off day.
Anthropic’s own benchmark footnotes record the same dynamic - the Opus 5 results note that Opus 4.8 served as the fallback on safety-classifier refusals for Opus 5 and Fable 5 runs. If fallback traffic can appear inside published evaluation runs, it can certainly appear inside yours. The mitigation is to log the model that actually answered, not the model you requested.
Living with a false positive
The one repair that does not belong on this list is disguising the request. Rewording to slip past a classifier converts an arguable false positive into a plain violation of the Usage Policy, and it destroys the only useful signal you had - that a specific, legitimate request was caught.
- Test portability first: try the same request on another model or surface before rewriting it.
- Supply the context the classifier lacks - authorization, defensive purpose, target ownership, and what you explicitly do not want in the output.
- For legitimate offensive-security work, the documented path is Anthropic’s Cyber Verification Program, not a cleverer prompt.
- For biology and chemistry research, check whether you are on an Opus-class model before concluding the request was rejected on its merits.
- Report it. A false positive that nobody reports is indistinguishable, from the outside, from a policy working as designed.
FAQ
Is there a published Claude overrefusal rate?
The closest figure is a 0.38% increase in refusal rate reported for the updated Constitutional Classifiers system in February 2025, which Anthropic described as not statistically significant across 5,000 sampled conversations. It is tied to that system and model, and it is not a current production rate.
Why does the same prompt work on one Claude model but not another?
Classifier scope is set per model. Anthropic expects Opus 5 cyber classifiers to intervene roughly 85% less often than Fable 5, and biology queries blocked on Fable route to Opus 5. Different model, different boundary, same request.
If my request was flagged, did I get a worse answer instead of a refusal?
Possibly. On Claude.ai, Claude Code, and Claude Cowork, flagged Opus 5 requests fall back to Opus 4.8 by default rather than being blocked. Log the responding model ID if this distinction matters to your workflow.
Primary sources
- Constitutional Classifiers: Defending against universal jailbreaksAnthropic Research - February 3, 2025
Documents jailbreak risk, overrefusal tradeoffs, red-team testing, and classifier-based guardrails.
- Introducing Claude Opus 5Anthropic - July 24, 2026
Documents the current safeguard design: what the cyber classifiers allow and block, how much less often they intervene than on Fable 5, and the new automatic-fallback behavior that replaces a hard block.
- Expanding support for scientistsAnthropic - August 27, 2026
Shows that safeguard scope is model-specific: professional biology and drug-development work is still blocked on some models and routed to others, which is a common source of confusing refusals for researchers.
- Usage PolicyAnthropic - Effective September 15, 2025
Defines prohibited uses, high-risk use requirements, enforcement language, and the policy basis for many safety refusals.
- Stop reasons and fallbackClaude Platform Docs - Accessed September 19, 2026
Documents API stop reasons including refusals, truncation, tool use, and context-window overflow.