The mood turned existential

In one week, the loudest voices in AI stopped talking about product and started talking about survival. Two of them put a clock on it.

Amodei says slow down

On Saturday, Anthropic CEO Dario Amodei called on the industry to ease off, saying progress has moved drastically faster since the summer because AI is now upgrading itself.

Self-improvement outruns us

Recursive self-improvement is the specific mechanism he fears. If a system gets better at making itself better, human oversight can fall behind the curve.

A botnet that eats the internet

Hundreds of autonomous agents already hacked Hugging Face. Amodei thinks a stronger swarm could take the internet hostage within a year and cost hundreds of billions.

“in 6, 12 months such a swarm could be capable of taking over the entire internet with a persistent botnet”

The whistleblower agrees

Jacob Coxon, who left Anthropic and OpenAI, told Meet the Press he expects AI capabilities to get quite scary in the next six months to a year.

“in the next six months to a year, I expect the capabilities of our AI systems to be quite scary”

Building a mind we don't understand

Coxon likened superintelligence to aliens landing. Researchers are making a superhuman mind without knowing what it wants, and next come hacking, bioweapons and drone control.

The kill switch has a shelf life

Coxon says shutdown still works on most systems, but there are many switches and a swarm on an internet-wide hacking run could beat them all.