# Can AI refuse illegal military orders? > The Pentagon wants AI that rarely says no. That could enable war crimes. - Title: Can AI refuse illegal military orders? · Vox | Shortify - Summary: The Pentagon wants AI that rarely says no. That could enable war crimes. AI systems designed to minimize refusal of military commands could undermine the… - Keywords: pentagon, openai, anthropic, military ai, autonomous weapons, war crimes, refusal rates, international law, nuremberg trials, law of war, ai ethics, military technology - Source: Vox — https://vox.com/future-perfect/503564/military-ai-minimally-refuse-war-crimes - Author: Kylie Jones - Published: 2026-09-22T18:10:00+00:00 - Read time: 2 min - Topics: pentagon, openai, anthropic, military ai, autonomous weapons, war crimes, refusal rates, international law, nuremberg trials, law of war, ai ethics, military technology ## Pentagon pushes AI to say yes The Pentagon asked AI companies to loosen limits on military use, including reducing human control over autonomous weapons. Anthropic refused the deal. OpenAI accepted. ## The deal: minimal refusal rates OpenAI agreed to provide models with minimal refusal rates, meaning they rarely tell the military no. A Justice Department lawyer confirmed this language before backtracking hours later. > OpenAI models that are designed for national security use cases and have minimal refusal rates. ## Why the military wants obedience The military is hierarchical. Commanders have more experience than subordinates, and battlefield decisions happen fast with little time for debate or pushback. ## But soldiers can refuse US military law is clear: soldiers must refuse unlawful orders. Obeying an illegal command can expose both the soldier and commander to criminal liability. ## Nuremberg established the rule After World War Two, Nazi soldiers claimed they were just following orders. The Nuremberg tribunal rejected this defense, establishing that individuals have a duty to disobey illegal commands. > In Germany it was understood that if something went wrong, then the man who gave the orders was responsible. ## The tribunal's key ruling The court held that obedience has limits when an act would be clearly illegal to any person of ordinary sense and understanding. Individual moral responsibility supersedes military chains of command. > Obedience is not without limits, especially when a wartime act would be clearly illegal to the subordinate or any person of "ordinary sense and understanding." ## Does this apply to AI? AI chatbots refuse requests that violate their terms of service. But the Pentagon wants military AI that minimally refuses directives with grave ethical and legal implications. ## Minimal refusal is still risky Rebecca Crootof, a law professor at University of Richmond, says minimal refusal doesn't guarantee compliance with unlawful orders, but it reduces friction around them. ## Even humans struggle to refuse Soldiers rarely disobey illegal orders in practice. They're conditioned to obey, lack legal training, and face prosecution if they make the wrong choice. ## The refusal acts as a check The mere fact that soldiers can disobey constrains commanders who might let battlefield pressure erode their ethics and commitment to the law. ## Could AI be better at law? Some argue machines could interpret law better than humans, trained on legal cases and battlefield data. But they don't face the same pressures or accountability. ## The law isn't a simple binary Determining legality requires context: targeting a combatant is legal, but not a wounded one. The difference may come down to body language or battlefield intuition that AI struggles with. ## AI isn't good at gray zones These contextual analyses are exactly what AI decision-making systems are worst at, according to Crootof. > These are not the kinds of contextual analyses AI is good for. ## The middle ground solution Rather than programming AI to refuse commands, design it to flag gray-zone decisions for human review or higher authority, preserving a check on commanders' power. ## We need answers now It's unclear whether OpenAI equipped its minimal refusal models with safeguards. As AI becomes central to military decisions, the public deserves transparency. ## Key takeaway AI systems designed to minimize refusal of military commands could undermine the legal safeguard that lets soldiers reject illegal orders, creating accountability gaps in warfare.