AI nearly triggers military strike

Military aircraft were already airborne this spring when U.S. officials discovered the intelligence driving an armed operation against a Chinese vessel was fabricated by an AI chatbot. The operation was aborted at the last minute.

“The operation was aborted at the last minute, narrowly averting a potential conflict with China”

How the false intelligence spread

A Special Operations Command analyst used an AI chatbot to synthesize open source and classified data, but the system misidentified a ship's cargo manifest. The analyst then used the tool again to format the false findings into an official-looking summary that circulated across command channels.

The speed problem

The Pentagon wants AI to accelerate decision-making and maintain advantage over China by speeding up response times. But that same speed allows hallucinations to travel up the chain of command before human verification can catch them.

Life and death consequences

Military decisions about targeting, intelligence analysis, and operational planning have life and death consequences. Service members need to understand that AI language models are inherently uncertain, especially when those decisions could lead to use of force.

“There are life and death consequences for those decisions.”

Safeguards, not avoidance

Experts say the military should add more oversight to AI rather than reject it entirely. Prioritizing adoption speed over safeguards will erode trust in these systems and ultimately slow their integration.

“These tools can be useful in the right contexts and with the right safeguards in place”