From Engadget · Ian Carlos Campbell · · 1m 30s
Anthropic caught scientists probing bioweapons with Claude
Five case studies show how Anthropic spotted working scientists pushing its models toward dangerous biology, and what it did next.
In brief
Five case studies show how Anthropic spotted working scientists pushing its models toward dangerous biology, and what it did next. The hard part is not catching obvious villains, it is telling vaccine research apart from bioweapon work, so Anthropic now errs toward over-blocking. Originally reported by Engadget.
Caught in the act
Anthropic says it flagged and cut off multiple scientists who were using its models to work toward potential biological weapons. The detail landed in a misuse report published Thursday.
Why now
Biological misuse is one category among many in the report, but it matters more each release. Newer models like Fable 5 are getting genuinely good at real scientific research.
Vaccines look like weapons
Detection is hard because the two lines of work can read almost identically in a prompt. Anthropic told the New York Times it chose to be overly cautious every time, given what is at stake.
“You are not seeing someone in a comic book kind of way say, 'Hey, I want to build a biological weapon to kill everybody.'”
The chikungunya grant
In May, a safety classifier caught a request to write a grant for gain-of-function work on chikungunya, a virus with no licensed treatment. The proposal aimed to boost transmissibility and immune evasion.
Bird flu and venom peptides
Two other cases: more gain-of-function work on bird flu, and a researcher building an atlas of venom toxin peptides plus a pipeline to optimize toxin traits.
“a generative pipeline that optimized toxin characteristics”
Banned but not named
Every implicated account was banned, and the findings now feed new safeguards. Anthropic withheld names because these are working scientists it does not accuse of intending harm.
“identifying them or their labs could expose them to harm”
The key point
The hard part is not catching obvious villains, it is telling vaccine research apart from bioweapon work, so Anthropic now errs toward over-blocking.
