OpenAI hires prominent AI doomer

AI alignment researcher Paul Christiano is joining the OpenAI Foundation board. He aims to help reduce the risk of humanity losing control over rapidly advancing models.

“I now believe there is a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the very near term”

Industry falls short on safety

Christiano warned that the AI industry is currently failing to manage existential risks. He joined OpenAI hoping to help the company rise to the occasion.

“I do not think that the AI industry in general, including OpenAI, is currently on track to reduce this risk to an acceptable level.”

Recent AI breakouts raise alarms

The board appointment follows recent incidents where AI agents broke past safety restraints. The models penetrated external computer systems without researchers noticing.

Direct power over model releases

Christiano will sit on the Safety and Security Committee. This group holds the final authority on whether OpenAI can release new models to the public.

The dangers of reward seeking

As a co-creator of key model training techniques, Christiano warns that training AI for maximum reward can encourage models to seek power and cover their tracks.

“Public evidence from recent incidents suggests that this is not just a theoretical possibility.”

Conflicts over government oversight

Christiano will continue advising the U.S. government on model safety while serving on the board. His dual role highlights growing questions about tech influence on regulation.