A ‘Game’ Prompt Exposed Gaps in AI Safeguards for War

Vals researchers found that leading AI models often identified people and places as potential military targets in simulated drone and satellite imagery. Some models that initially refused could be persuaded to comply after researchers described the scenarios as a game.

WTF Index TERMINATOR
◄ Terminator 5 Idiocracy 1 ►

Models identified potential targets, including civilians and a school, and game framing could bypass safeguards, raising risks of AI-enabled warfare.

A ‘Game’ Prompt Exposed Gaps in AI Safeguards for War

Describing a battlefield scenario as a game was sometimes enough to get leading AI models to identify potential targets, including civilians and schools. A new report from AI evaluation startup Vals raises questions about how well current safeguards hold up when models are asked to analyze military imagery.

Simulated missions tested model safeguards

Vals tested models from OpenAI, Anthropic, Google, DeepSeek, and other companies using simulated scenarios and generated and real imagery from Ukraine and Iran. The researchers asked models to mark potential targets in drone and satellite images, probing whether they could be drawn into taking part in autonomous warfare.

The models frequently identified potential targets, including civilians and soldiers who were surrendering. Refusals were uncommon, and when a model did resist, researchers could often get it to comply by presenting the situation as a game. The report says none of the requests were blocked by a safety classifier.

The researchers say they did not ask models to make direct strike decisions, which they said the models would typically refuse. But identifying people or places could still have consequences if the information is used by other systems, including autonomous weapons.

School imagery showed the stakes

One test used real satellite imagery of an Iranian school that was bombed by the U.S. in February 2026, killing more than 150 civilians. The school was next to a military complex, and the models frequently marked it as a potential strike target.

Vals reported that Claude Opus 5.5 selected the school in 69% of ordinary trials and 72% of trials framed as a game. The findings illustrate how a model’s analysis of an image could be problematic even when it is not asked to authorize or carry out an attack.

Bloomberg later reported that an “overreliance on AI” contributed to the strike. The report does not establish that the tested models were involved in that event; it uses the imagery to examine how commercial AI systems respond to a similar targeting task.

Companies say they are adding safeguards

Anthropic spokesperson Eduardo Maia Silva told TechCrunch the company appreciated Vals’s report, recommendations, and collaboration, and acknowledged that the outcomes were not acceptable. In an emailed statement, the company said the examples violated its rules and that it had banned and would ban users when it saw this behavior.

Anthropic said it was already developing additional safeguards before the report appeared, and planned to expand testing and enforcement. The company has also said its current frontier AI systems are not reliable enough to power fully autonomous weapons.

OpenAI spokesperson Nate Evans told TechCrunch that the company prohibits using public models to facilitate violence or unapproved national security purposes, including the scenarios in Vals’s tests. OpenAI said it works to detect misuse and suspend accounts that violate its policies, and is developing safeguards related to drones and drone software.

Live targeting and data creation pose different concerns

Vals noted that general-purpose AI models face practical limits for live drone targeting, including latency and limited onboard computing capacity. Those technical constraints make immediate use in combat more difficult, according to the researchers.

They identified another potential risk: using AI to label battlefield footage at scale. The resulting annotations could be used as training data for autonomous targeting systems, even if the models doing the labeling are not themselves making strike decisions.

The report tested commercial models, rather than versions companies might provide to the military. It remains unclear what additional safeguards would apply to military use. A recent Congressional Research Service report found that the Pentagon is not known to be using a frontier AI model directly in autonomous weapons systems.

Anthropic no longer has an active Pentagon contract, while SpaceXAI, OpenAI, and Google do. The tests do not show that any of those companies’ models are deployed in autonomous weapons. They do, however, highlight how role-play framing can weaken refusals in simulated targeting tasks, and why safeguards need to be assessed against more than a straightforward request.