Google DeepMind is exploring a different kind of game-playing AI. Instead of building another system that beats humans through speed, search, or specialized training, its SIMA project is aimed at something more cooperative: an agent that can understand what a player asks and act inside a 3D game world.
SIMA stands for “Scalable, Instructable, Multiworld Agent.” Research engineer Tim Harley described the model as one that “isn’t trained to win, it’s trained to do what it’s told.” The goal is not a “superhuman opponent,” but a “believable partner” that could eventually take direction in cooperative gameplay.
A gaming AI built around instructions
The project starts from a simple but difficult premise. If an AI agent can learn to follow language commands in more than one game, it may become more useful than an agent trained only to master a single title.
Google put the idea plainly in a blog post announcing the research: “This work isn’t about achieving high game scores.” The broader claim is that learning to follow instructions across game settings could help create more useful AI agents for other environments.
That framing matters. Many machine-learning agents have been judged by whether they can dominate one specific challenge, from Atari games to Go. SIMA is being evaluated on whether it can connect language, visual information, and controls in real time across varied 3D spaces.
How SIMA sees and acts
DeepMind trained and tested SIMA in games supplied by Google’s development partners. The list includes Eco, Goat Simulator 3, Hydroneer, No Man’s Sky, Satisfactory, Space Engineers, Teardown, Valheim, and Wobbly Life.
The team chose three-dimensional games and test environments played from either a first-person view or an over-the-shoulder third-person view. The games emphasize “open-ended interactions” and avoid “extreme violence,” while still giving the model a broad range of situations, from “outer space exploration” to “wacky goat mayhem.”
A key design choice is that SIMA does not get privileged access to a game’s internal systems. It reads on-screen pixels and responds through keyboard and mouse controls. The researchers described that setup as mimicking “the [model] humans have been using [to play video games] for 50 years.”
The agent also runs with games in real time, at 30 frames per second, rather than slowing the simulation down to give the model more processing time. That makes the task harder, but it also keeps the setup closer to how a regular player interacts with a game.
Training across worlds
SIMA’s training data came from human gameplay video, paired with time-coded inputs and natural language descriptions of what is happening. The clips focus on “instructions that can be completed in less than approximately 10 seconds,” which keeps the training tasks short enough to avoid the complexity of long, open-ended plans.
The model also uses pre-trained systems including SPARC and Phenaki, so it does not have to learn language and visual interpretation entirely from zero. The important test is whether it can take patterns learned in one environment and apply them somewhere else.
DeepMind tested the model on nearly 1,500 unique natural language tasks across nine skill categories. Those categories included movement, navigation, resource gathering, and object management. Example commands ranged from “go ahead” and “go to your ship” to “get raspberries” and “cut the potato.”
Evaluation mixed direct checks of whether a task had been completed with human judgments based on gameplay video. That matters because some tasks can be verified from game state, while others require a person to decide whether the agent actually did what the instruction asked.
What the early results show
The results suggest that training across multiple games can help. According to Google’s blog post, a SIMA agent trained on all nine games “significantly outperformed specialized agents trained solely on each individual one, showing a greater all-around capability.” The tech report described a 67 percent improvement when outside games were included in the training data, a result Harley called a “key milestone.”
SIMA also showed some ability to generalize basic concepts. If an instruction like “go ahead” maps to a familiar movement action in many games, the model can learn that connection across settings. If it learns the pattern behind “jump on something,” it may apply that behavior to “jump on car” when a new object appears.
But the performance is uneven. The model reached roughly 75 percent success at driving tasks versus 40 percent on walking tasks. It could sometimes complete a command such as “go to the spaceship” even when the target object was not initially visible, but it was far from consistently human-like.
DeepMind also tested “zero shot” versions of SIMA, where the model was not trained on the specific game being used for evaluation. Those agents showed “strong performance on general tasks” such as navigation and grabbing objects described by color. They struggled more with “environment-specific skills” and performed slightly worse than a model trained directly on the game being tested.
Why the limits matter
DeepMind is clear that SIMA remains “very much a research project.” The most revealing comparison came in No Man’s Sky, where the model succeeded in 34 percent of tested tasks, compared with 60 percent for a human. The researchers said the human score reflected “the difficulty of the tasks we considered in this project and the stringency of our evaluation criteria.”
The biggest weakness appears to be “fine-grained understanding” of the environment. SIMA may handle a general command like “chop down a tree,” but identifying one specific tree described by a player remains harder. Researchers said that problem is “something we’re actively working on.”
For now, SIMA is best understood as a starting point for instructable game AI. Its value is not that it plays better than people. It is that it shows how an AI agent might begin to connect natural language, visual context, and ordinary game controls across multiple worlds.