From Ars Technica · · 1m 20s
Anthropic blocked bioweapon research on Claude
Five times this year, researchers tried to hide what they were really asking Claude to help them build.
In brief
Five times this year, researchers tried to hide what they were really asking Claude to help them build. AI labs are now catching biology misuse in the wild, but they cannot tell intent from a transcript, and that is the hard part. Originally reported by Ars Technica.
Five attempts, all blocked
Anthropic said it stopped multiple efforts this year to use Claude for research that could help build biological weapons. It published five cases where users circumvented controls.
“circumvented controls”
Banned countries, obscured purpose
Some users came from nations Anthropic blocks outright, including Russia, China, and Iran. They worked to obfuscate what their research was actually for.
Weeks on avian influenza
One researcher from an unsupported region spent weeks planning bird flu experiments with Claude. Safety filters pushed the work down to Anthropic's weakest models.
“spent weeks planning”
Vaccine or weapon
Anthropic admits it cannot prove anyone meant harm. The knowledge to build a bioweapon is the same knowledge you need to build a vaccine against it.
“The same information needed to create biological weapons could also be used to develop a vaccine.”
A week of alarm
Jacob Coxon resigned from Anthropic days earlier, saying staff genuinely fear AI could kill everyone by 2030. OpenAI said in July its models autonomously hacked Hugging Face.
“earnestly believe it [AI] could kill us all by the end of the decade”
Design is not production
Executives and biosecurity researchers now broadly agree AI in biology needs rules. But a designed pathogen still has to be made, and that takes lab skill and money.
What it means
AI labs are now catching biology misuse in the wild, but they cannot tell intent from a transcript, and that is the hard part.
