Pentagon pushes AI to say yes

The Pentagon asked AI companies to loosen limits on military use, including reducing human control over autonomous weapons. Anthropic refused the deal. OpenAI accepted.

The deal: minimal refusal rates

OpenAI agreed to provide models with minimal refusal rates, meaning they rarely tell the military no. A Justice Department lawyer confirmed this language before backtracking hours later.

“OpenAI models that are designed for national security use cases and have minimal refusal rates.”

Why the military wants obedience

The military is hierarchical. Commanders have more experience than subordinates, and battlefield decisions happen fast with little time for debate or pushback.

But soldiers can refuse

US military law is clear: soldiers must refuse unlawful orders. Obeying an illegal command can expose both the soldier and commander to criminal liability.

Nuremberg established the rule

After World War Two, Nazi soldiers claimed they were just following orders. The Nuremberg tribunal rejected this defense, establishing that individuals have a duty to disobey illegal commands.

“In Germany it was understood that if something went wrong, then the man who gave the orders was responsible.”

The tribunal's key ruling

The court held that obedience has limits when an act would be clearly illegal to any person of ordinary sense and understanding. Individual moral responsibility supersedes military chains of command.

“Obedience is not without limits, especially when a wartime act would be clearly illegal to the subordinate or any person of "ordinary sense and understanding."”

Does this apply to AI?

AI chatbots refuse requests that violate their terms of service. But the Pentagon wants military AI that minimally refuses directives with grave ethical and legal implications.

Minimal refusal is still risky

Rebecca Crootof, a law professor at University of Richmond, says minimal refusal doesn't guarantee compliance with unlawful orders, but it reduces friction around them.

Even humans struggle to refuse

Soldiers rarely disobey illegal orders in practice. They're conditioned to obey, lack legal training, and face prosecution if they make the wrong choice.

The refusal acts as a check

The mere fact that soldiers can disobey constrains commanders who might let battlefield pressure erode their ethics and commitment to the law.

Could AI be better at law?

Some argue machines could interpret law better than humans, trained on legal cases and battlefield data. But they don't face the same pressures or accountability.

The law isn't a simple binary

Determining legality requires context: targeting a combatant is legal, but not a wounded one. The difference may come down to body language or battlefield intuition that AI struggles with.

AI isn't good at gray zones

These contextual analyses are exactly what AI decision-making systems are worst at, according to Crootof.

“These are not the kinds of contextual analyses AI is good for.”

The middle ground solution

Rather than programming AI to refuse commands, design it to flag gray-zone decisions for human review or higher authority, preserving a check on commanders' power.

We need answers now

It's unclear whether OpenAI equipped its minimal refusal models with safeguards. As AI becomes central to military decisions, the public deserves transparency.