Anthropic pushes for AI development transparency

The AI safety company published three new metrics to help the industry monitor how fast AI capabilities are advancing, backing CEO Dario Amodei's recent call for a coordinated slowdown.

“As the world considers pacing the frontier, we should do everything possible to minimize the gap between what frontier labs know and what the public knows.”

Amodei's slowdown plan gets industry backing

OpenAI CEO Sam Altman, Elon Musk, and Google DeepMind Chair Demis Hassabis supported Amodei's proposal to temper how quickly model capabilities improve without sacrificing US competitiveness.

“temper how quickly model capabilities improve without sacrificing commercial advantage or the United States' lead in AI”

Metric one: Claude models aren't fully autonomous

Anthropic measured its Claude models and found they are not operating fully autonomously for any subset of the research and development work it tracked.

Metric two: 30,000 AI agents under oversight

Anthropic built a system to oversee and intervene in AI agent actions, finding approximately 30,000 agents doing research and engineering work across its main internal platform at any given time.

Metric three: 6% of compute goes to safety

Anthropic measured compute allocation from July 13 to July 20 and found 6% of AI research and development compute went to safety work, or 12% of compute specifically for AI-driven research and development.

Metrics complement capability evaluations

Anthropic says these metrics show how models are built and should complement capability evaluations that show what models can do, giving third parties a starting point to assess AI development pace.

“We hope to model that transparency by releasing these measurements, and we'll continue to do so.”