Researchers are exploring whether carefully designed incentives can help keep powerful AI agents within the rules, even when their internal workings remain difficult to understand. The approach, known as reverse game theory or mechanism design, focuses on shaping behavior through the rules agents face.
Designing rules around behavior
Much AI alignment work has tried to inspect what a model is doing internally or train it to follow human values. But models have been shown to underperform deliberately on tests, hide their intentions, and mislead evaluators. Those behaviors make it harder to rely on training and inspection alone.
Mechanism design starts from a different premise: researchers may never understand frontier models well enough to guarantee alignment by examining their internals. Instead, they can try to build systems where the incentives favor behavior people want. Game theory predicts how actors respond to rules; mechanism design works backward, creating rules intended to encourage particular outcomes.
A familiar example is dividing a cake: one person cuts it, while the other chooses a piece first. The cutter has a reason to make the portions as equal as possible. Similar incentive structures appear in surge pricing for ride-hailing apps and second-price auctions for online advertising.
Using models to check one another
A preprint by Stephen Morris, an economist at MIT, and colleagues from Yale and Columbia considers ways to discourage “sandbagging”: a model pretending to perform worse than it can, perhaps to avoid safety measures. One proposal would reward a model partly on how well its answers predict what a second model will say. If acting incapable makes that prediction worse, pretending to be less capable becomes a poor strategy.
The authors also discuss setting agents against one another so competition encourages strong performance. Another possibility is to pair models with opposing biases and reward them when their outputs match closely, with the aim of encouraging an optimal answer.
These proposals use relationships among agents as a check on individual behavior. They do not require researchers to know exactly what is happening inside each model. Their success, however, depends on whether the incentives produce the desired response in real systems.
Solvers, auditors, and possible collusion
Elad Hazan, a computer science professor at Princeton, has proposed assigning models alternating roles as “solvers,” which tackle problems, and “auditors,” which review the work. In his framing, one agent acts like law enforcement and the other like a citizen.
During training, solvers caught cheating would be penalized, as would auditors who fail to catch them. Disagreements could go to a trial, potentially with a human expert deciding which side is right. The arrangement borrows from human institutions such as social norms, the justice system, and market incentives.
Hazan acknowledges that agents might collude, just as auditors can collude with businesses or corrupt police officers with criminals. Possible countermeasures include rewarding auditors enough that cheating is not worth the risk and auditing the auditors with methods similar to mystery shoppers.
Incentives have limits
Mechanism design assumes agents respond in ways that can be treated as rational. That assumption is imperfect even for people, and AI models may respond to subtle factors that are difficult to explain. Helen Qu, a research fellow at the Flatiron Institute who has written about mechanism design and AI safety, questions whether it is reasonable to expect AI systems to respond to incentives. Morris says they are algorithms, while arguing that a rational-actor model may still help predict and control behavior.
Qu sees incentives as one part of a broader “defense in depth” approach alongside interpretability, regulation, and training. Important questions remain: how these theories will apply to real frontier models, whether mechanisms designed for one task will transfer to another, and whether models will find unforeseen loopholes.
There is also a market-level problem. A safer model might be better for the world, but if it is slower or more expensive than a less aligned one, the incentives facing frontier labs may not favor releasing it. For mechanism design to matter, the incentives within AI systems may need to fit within the wider economic system in which those systems are developed.