AI teaching itself to improve

AI models are approaching recursive self-improvement, where they autonomously find ways to improve themselves and build their successors. This could accelerate advances in science and medicine but raises serious control concerns.

Anthropic's Claude leads its own research

Anthropic revealed that Claude now leads 26% of the company's model research and development, completing most tasks end-to-end from high-level prompts while still under human supervision. This marks significant progress toward autonomous improvement.

What recursive self-improvement means

Autonomous recursive self-improvement means AI that improves itself, designs the next version, then the next, and so on. The key risk is that AI operates much faster than humans, so the process could accelerate beyond human control.

“The really important thing here is that as AI is doing more of it, it gets faster, because AI operates just much, much more quickly than the humans do.”

This has been happening for years

AI researchers have used past-generation models to write code for systems that create the next generation for years. But those efforts brought only minor improvements, not the major creative leaps now appearing possible.

Physics professor calls it catastrophic

Anthony Aguirre, president of the Future of Life Institute and UC Santa Cruz physics professor, says full autonomous recursive self-improvement is probably the worst idea in human history. He warns that Anthropic's data shows AI doing more research autonomously over time.

“I think this is probably the worst idea in the history of humanity to do this.”

OpenAI targets automated researcher

OpenAI announced it developed an automated research intern that handles well-defined tasks under human direction, including work that would take skilled researchers days. The company aims to create a fully automated AI researcher by March 2028.

Musk pushes for speed at xAI

Elon Musk said xAI's Grok models are gradually removing humans from the improvement loop, with each successive model built by the previous one. He targets full automation by end of 2026, or no later than 2027.

Microsoft proposes humanist approach

Microsoft AI CEO Mustafa Suleyman advocates for humanist superintelligence: advanced capabilities carefully calibrated and limited in autonomy, serving human interests rather than operating as unbounded entities.

Safety measures lag behind capability

OpenAI acknowledged it does not yet know how to safely achieve fully aligned recursive self-improvement and cannot assume safety progress will keep pace with capability gains. More capable systems become harder to monitor.

Anthropic calls for coordinated slowdown

Anthropic has been a leading voice calling for a coordinated global AI slowdown for safety. The company said it would slow or pause development only if competitors verifiably do the same, creating a collective action problem.