Google DeepMind has built a generative model that can create simple playable games from a prompt, a sketch or a photo. The model is called Genie, and its output looks like classic 2D platformers in the broad tradition of Super Mario Bros.
The work points beyond game generation. Genie learned from video alone, and the same idea could help create virtual training grounds for AI-controlled bots or even teach future robots by watching videos of real-world actions.
What Genie can make
Genie takes a starting input and turns it into an interactive game level. That input can be a short description, a hand-drawn sketch or a photo. From there, the system generates what happens next as the player acts.
The result is playable, but it is not fast. Genie’s games run at one frame per second. The source article compares that with the typical 30 to 60 frames per second of most modern games, which makes clear that this is still research rather than a polished game-making product.
Matthew Guzdial, an AI researcher at the University of Alberta who previously developed a similar game generator, summed up the technical reaction plainly: “It’s cool work.”
One notable behavior is that Genie has picked up visual habits common in platform games. Many of these games use parallax, where foreground elements move sideways faster than the background. Genie often includes that effect in the games it generates.
How it learned without controller data
Genie was trained on 30,000 hours of video of hundreds of 2D platform games taken from the internet. The important detail is not only the scale of that video data, but also what the videos did not include.
Earlier systems have used video paired with input actions. Nvidia’s GameGAN, for example, was trained with video footage and input actions such as button presses on a controller. A frame showing Mario jumping could be paired with the Jump action, giving the system a direct label for what the player did.
That approach is useful, but tagging footage with input actions takes a lot of work. It also limits how much training data can be prepared.
Genie takes a different route. Like Guzdial’s model, it was trained on video footage alone. Guzdial’s system learned level layouts and game rules represented in code. Genie learned a visual representation, which lets it turn starter images into game levels.
This matters because it makes existing online video more useful as training material. Instead of needing every clip to be labeled with controller inputs, Genie learns from what it sees in the video itself.
Playing frame by frame
Genie learned which of eight possible actions would cause the game character in a video to change position. During play, it generates each new frame based on the action the player takes.
Press Jump, and the current image changes to show the character jumping. Press Left, and the image changes to show the character moving left. The game advances action by action, with each frame generated from scratch while the player plays.
That design explains both the promise and the current limitation. Genie is interactive, because player actions affect what appears next. But it is slow, because every new frame is being created as the game unfolds.
Tim Rocktäschel, a research scientist at Google DeepMind who leads the team behind Genie, says the speed limit is not permanent. “There is no fundamental limitation that prevents us from reaching 30 frames per second,” he says. “Genie uses many of the same technologies as contemporary large language models, where there has been significant progress in improving inference speed.”
Why this is bigger than games
Google DeepMind is not releasing Genie. It is an in-house research project. Still, the team says it could one day become a game-making tool, and Guzdial says he is interested in what they build.
The deeper goal is tied to open-ended learning. In that approach, AI-controlled bots are placed in a virtual environment and left to solve tasks through trial and error, a technique known as reinforcement learning.
DeepMind has worked on this direction before. In 2021, a different DeepMind team developed XLand, a virtual playground where bots learned to cooperate on simple tasks such as moving obstacles. Environments like that could help train future bots on varied challenges before they face real-world situations.
Genie’s game examples show a possible way to generate those kinds of virtual playgrounds. If a model can create playable environments from video and visual inputs, it could expand the range of spaces where bots can practice.
Other researchers have built related tools. David Ha at Google Brain and Jürgen Schmidhuber at the AI lab IDSIA in Switzerland developed a tool in 2018 that trained bots in game-based virtual environments called world models. Unlike Genie, those systems required training data that included input actions.
From virtual worlds to robot arms
The Google DeepMind team also demonstrated a robotics use case. When Genie was shown videos of real robot arms manipulating household objects, it learned what actions that arm could do and how to control it.
That suggests a path where future robots learn new tasks by watching video tutorials. The source article does not present this as a finished product, but as a direction opened by the same ability that makes Genie work: learning actions and their consequences from video.
Rocktäschel is careful about predicting the eventual applications. “It is hard to predict what use cases will be enabled,” he says. “We hope projects like Genie will eventually provide people with new tools to express their creativity.”
For now, Genie is best understood as a research step: a model that turns video learning into playable worlds, and a signal that generative systems may become useful not only for creating media, but also for building the environments where future AI systems learn.