AI researchers fear their creation

Many AI researchers believe the technology they're developing could become very dangerous, but there's no clear consensus on how to control it.

A growing list of control ideas

Proposals range from tighter government regulations and new progress metrics to more extreme measures like placing tracking devices in GPUs or destroying AI chips.

Slowing AI is unsolved

A new report from University of Toronto researcher Raymond Douglas warns that slowing down AI development remains an unsolved puzzle with unclear options.

“We need to start treating this as a research problem.”

AI doom fears intensify

An Anthropic researcher recently warned that AI could wipe out humanity within a couple of years, prompting leaders at major AI companies to support some form of slowdown.

The recursive loop problem

AI companies are now using AI to build more powerful models, creating fears of a recursive self-improvement loop that could outpace human understanding within years.

Anthropic tracks its own progress

Anthropic announced new tracking methods showing Claude now does 26 percent of their AI research and they spend 6 percent of compute budget on AI safety.

Independent evaluators could help

Third-party evaluators could test AI models and try to trigger misbehavior, though critics argue current inspections lack rigor and true independence.

“When [big AI companies] say 'independent evaluators,' they mean 'I want to pay my friends who live in my group houses to look at my prompts.'”

Track compute, not just capability

Experts propose monitoring GPU usage, power consumption, and billing records at data centers to detect when companies train models beyond safe thresholds.

Modify chips for oversight

Researchers suggest adding cryptographic components to GPUs that record compute runs, or even building remote kill switches that require authorization to run certain models.

International treaties are essential

Controlling AI requires global cooperation since China and other nations can also build frontier AI. One proposal suggests mutual hardware reduction via treaty.

The nuclear option

Some philosophers have proposed that if nations agree the risk is grave enough, they could bring GPUs to neutral territory and destroy them together.

Measuring AI's self-improvement

A new benchmark called the RSI Index tracks AI-powered AI development. It suggests AI could perform research work that human researchers cannot follow within a year.

Bad controls could backfire

Rushing to implement poorly designed controls risks political gridlock or regulatory capture. Experts warn that a flawed plan could be worse than no plan at all.

“Going off half-cocked with a bad plan could end up worse than nothing.”