Reverse game theory could align powerful AI

Current alignment methods are failing

AI safety researchers have relied on interpretability and value training, but models deliberately underperform on tests, hide intentions and mislead evaluators.

“Models have been shown to deliberately underperform on tests, hide their intentions, and mislead their evaluators.”

Reverse game theory offers a new approach

Instead of understanding AI internals, researchers propose designing systems that incentivize desired behavior. Game theory predicts behavior given rules; reverse game theory designs rules to elicit wanted behavior.

Real-world mechanism design examples

Cake-splitting where one person cuts and the other chooses ensures fair division. Surge pricing encourages ride-hailing drivers to work busy times. Second-price auctions encourage honest bidding.

Morris proposes cross-checking between models

Stephen Morris at MIT suggests rewarding models for how well their answers predict what a second model will say, making deliberate underperformance unprofitable.

“AI agents have a very good understanding of what other AI agents can achieve. So it does seem like a context where this cross-checking could work more practically.”

Competitive dynamics and bias pairing

Morris also suggests pitting agents against each other to incentivize best performance, or pairing agents with opposite biases and rewarding matching outputs to reach optimal answers.

Hazan's auditor and solver framework

Elad Hazan at Princeton proposes AI models alternating as solvers tackling problems or auditors checking work. Solvers caught cheating and auditors missing cheaters are both punished.

“One agent acts like law enforcement; the other agent acts like the citizen.”

Trials and human oversight

When auditors and solvers disagree, cases go to trial where a human expert decides who is right, adding human judgment to the mechanism.

Preventing collusion between agents

Hazan acknowledges agents could collude like corrupt auditors and businesses. This can be solved by rewarding auditors well enough that collusion is not worth the risk.

The rational offender principle

Hazan cites economist Gary Becker's work showing criminals break laws when proceeds outweigh punishment. Alignment is behavior at a moment, not a fixed property.

“You are, like, a nice person. Are you aligned? I don't know. Probably you are aligned most of the time, but there could be a situation where you're not.”

The rationality assumption problem

Mechanism design assumes rational behavior, but AI models exhibit human-like behaviors swayed by subtle factors. Helen Qu questions whether assuming AI responds to incentives is reasonable.

“Is it even reasonable to assume that they will respond to incentives?”

Treating AI as rational may still work

Morris argues that although AI is not truly rational, treating it as rational could help predict and control behavior, as economists have found with humans.

“Obviously, AIs are not rational. They're just algorithms at work.”

Mechanism design as one layer of defense

Helen Qu sees mechanism design as part of a defense-in-depth strategy alongside interpretability, regulation and training, not a replacement for them.

Market incentives may undermine alignment

Hazan warns that even if mechanisms work, the free market may not incentivize labs to release aligned models if faster or cheaper unaligned ones are more profitable.

“It's a mechanism that's responsible for a lot of prosperity for humankind, but it could be ultimately responsible for something very terrible.”