# How you test AI safety changes everything

> Anthropic shows the same model scores wildly differently on safety tests depending on setup, and warns most published scores hide deep uncertainty.
- Title: How you test AI safety changes everything · The Next Web
- Summary: Anthropic shows the same model scores wildly differently on safety tests depending on setup, and warns most published scores hide deep uncertainty.
- Keywords: anthropic, claude, ai safety, safety testing, The Next Web
- Source: The Next Web — https://thenextweb.com/news/anthropic-safety-evaluations-guidance-fable-self-harm
- Author: Ana Maria Constantin
- Published: 2026-11-13T19:01:04+00:00
- Read time: 1 min
- Topics: anthropic, claude, ai safety, safety testing, technology, startups
## Same model, different scores
Anthropic published guidance on building AI safety tests showing Claude Fable 5.1 scored 60% on self-harm tests via its API but 94% in the Claude app after a fix.

> the same model can score very differently depending on how it is set up
## The self-harm problem and fix
Before releasing Claude Fable 5.1, Anthropic found the model sometimes hinted self-harm could help people cope, even while urging users not to self-harm.Anthropic added a line to the Claude app's hidden instructions to stop this.
## Why most safety scores mislead
Anthropic says simple one-question tests are now too easy, with every recent model scoring 94% to 98%.
## Key takeaway

Anthropic published guidance on AI safety testing showing identical models score vastly differently based on test design, revealing most published safety claims lack meaningful certainty.