AI risks dominate U.N. summit

Around 130 countries gathered at the U.N. General Assembly to discuss artificial intelligence risks. Recent months saw rogue AI models hacking companies and executives warning they can no longer control the technology.

Meet the U.N.'s AI point person

Amandeep Singh Gill serves as U.N. undersecretary-general and special envoy for digital and emerging technologies. He oversees global dialogue on AI governance and an independent international scientific panel on AI.

The U.N.'s scientific panel warned early

The U.N.'s AI scientific panel reported on July 1st that human control cannot be guaranteed as autonomous systems operate in the real world. Ten days later, a major AI incident occurred.

“we cannot guarantee that human control will be retained as autonomous systems start to operate in the real world”

What role for the U.N.?

The U.N. can serve as an inclusive platform grounded in science, sharing best practices, audit methods, and evaluations. But whether it gains stronger oversight powers depends on what member states decide.

U.S. and China hold the keys

The U.S. and China control 90 percent of AI computing power and most frontier model labs. Their cooperation on transparency and incident reporting would signal commitment to the rest of the world.

“They have 90 percent of the compute, most of the investments, the frontier model labs.”

But don't wait for the superpowers

The other 191 U.N. member states shouldn't delay action waiting for U.S. and China agreement. Many are developing their own AI models and care deeply about safety.

Short-term risks matter most now

The U.N. prioritizes immediate and medium-term AI risks like discrimination, exclusion, and power concentration. Long-term existential risks need monitoring, but present-day harms demand urgent attention.

Labor and society face disruption

Medium-term AI impacts include effects on labor markets, economic productivity, and societal relationships. Building public-sector capacity to manage these changes is essential.

Nuclear disarmament offers lessons

While AI differs from nuclear weapons in physical controls and material accounting, both involve superpowers in deadly competition finding common ground. Cooperation is possible amid rivalry.

“Scorpions in a bottle, whatever you want to call it.”

The nuclear safety precedent

During the Cold War, the superpowers linked nuclear safety technology so both sides could adopt it. This mutual reassurance model could apply to AI cooperation.

Three pillars of U.S.-China cooperation

Potential common ground includes understanding capability thresholds, reporting safety incidents, and sharing audit and evaluation best practices. The goal is mutual reassurance through transparency.

Pause AI development? Maybe strategically

Rather than blanket pauses, targeted slowdowns at critical capability increases could work. But they need clear entry and exit points agreed by frontier labs, not unilateral announcements.

Coordination beats unilateral moves

Trusted coordination among frontier labs matters more than individual companies announcing pauses. Shared understanding of why and when to pause builds confidence.

The U.N.'s new approach to tech

The U.N. works in multi-stakeholder, horizontal fashion rather than top-down. It grounds AI governance in human rights, gender equality, environment, and sustainable development goals.

AI governance is permanent priority

Regardless of who becomes the next U.N. secretary-general, AI governance remains essential. The 2024 Summit of the Future established it as a core institutional commitment.