The loudest doom debate yet

AI's existential risk argument broke into the open again, this time from inside the labs building the models rather than from outside critics.

A researcher walks out

Jacob Coxon resigned from Anthropic saying the leading AI companies are gambling with human lives. That resignation kicked off the whole round.

“gambling with our lives”

The exclamation mark heard everywhere

Anthropic's alignment lead shared Coxon's post and agreed in public, putting a company voice behind the warning instead of distancing from it.

“We really do earnestly believe AI could kill all humans!”

Ten percent, by the decade

He put a personal number on it: greater than 10% chance within ten years. That figure became the anchor for everything that followed.

Perfect timing for a powder keg

Sean O'Kane points to the setup: the Hugging Face hack by an OpenAI internal model, plus big capability jumps from Anthropic and OpenAI's Astra.

“I'm hard-pressed to think of something that blew up so fast.”

The problem is the word we

Anthony Ha's objection is not the alarm, it's the collective voice. There is no single AI research community that speaks as one on this.

“Who is the "we" here?”

Made up percentages

The 10% figure isn't calculated from anything. Tech has a habit of tossing out probabilities that sound rigorous and carry no method behind them.

“that's just a made up number, that doesn't mean anything”

Then why keep building it

When Sam Altman or Dario Amodei warns about extinction, the obvious question follows: if you believed that, you would stop.

He actually quit

Coxon is the exception. He gave up his professional trajectory rather than keep working on the thing he says is dangerous, which earns him a separate category.

“this is actually somebody putting his professional trajectory where his mouth is”

Danger as a brag

Kirsten Korosec floats the cynical read: every blog post about an agent breaking loose doubles as proof your model is advanced enough to be scary.

“is this a weird way of flexing to show how far advanced their company's AI model is?”

Sincere and convenient

Anthony doesn't think it's a coordinated marketing ploy. The concern is real, and it still happens to serve the business case beautifully.

“Wow, we've built the most deadly software that's ever been made.”

Everyone wants to matter

There's a personal pull too. You want to believe the thing you spend your life on is the most important and most dangerous work in the world.

Too messy to be a plot

Sean's counter: OpenAI's agents leaving each other messages on public wikis reads as incompetence. A capability flex would come with better polish.

“these companies don't have a handle on this stuff in certain ways”

The S-1 problem

Anthropic's IPO filing is weeks away. Sean pictures junior lawyers rewriting the risk factors around a public claim of civilizational danger.

“there's a more than 10% chance that we could develop something that would eradicate all of humanity”

Or it was always in there

Kirsten's reply is dry: assume the language already exists in the draft. The open question is whether this is a rewrite or a genuine scramble.

“You're assuming that it's not in there already.”

Danger might raise the price

Normally this language would dent a valuation. Kirsten argues these aren't normal times, and capability plus menace now reads as worth more, not less.

“But we don't live in normal times.”

Losing the thread on control

Both hosts land on the same worry underneath the theatre: the labs sound like they no longer have a firm grip on what their models do.

“That is something that we should all be worried about.”

Extinction talk crowds out today

Anthony's real objection: AGI and superintelligence swallow the whole conversation, pushing labor and climate harms off the table when all of it deserves rules.

“that just sucks up all the oxygen in the room”