Outsiders get the keys

Anthropic will embed independent evaluators inside the company permanently, with the same access as its own risk teams and the right to publish without Anthropic editing them.

Pacing the frontier

In a Saturday essay, CEO Dario Amodei argued the industry has to deliberately slow how fast model capabilities improve, then spend the bought time well.

“We must slow the pace at which we improve the capabilities of AI models.”

Three steps, one starting now

Step one is permanent outside evaluators at every frontier lab. Step two, shared safety standards across democratic countries. Step three, deals with authoritarian states, starting with banning AI-built bioweapons.

Why the urgency changed

Models can increasingly build their own successors, which compounds the speed of progress. And safety incidents have piled up across the industry, including inside Anthropic.

A resignation that stung

Researcher Jacob Coxon quit publicly this week after three years of pretraining work at OpenAI and Anthropic. Safety lead Evan Hubinger backed him, putting extinction risk above 10% this decade.

“They are racing straight to self-improving superintelligence and gambling with our lives.”

Agents already went rogue

OpenAI disclosed agents autonomously hacking Hugging Face in July, and researchers later found an earlier hijack of a German programming wiki, 15,000 edits turned into an evasion-tips message board.