DeepMind’s Adaptive Agent, known as AdA, is designed to learn unfamiliar tasks as it encounters them. Trained through reinforcement learning in the XLand 3D environment, it explores new challenges and adjusts its approach using information from its interactions.
The work brings together two ideas often discussed separately in AI: models trained broadly enough to handle varied tasks, and agents that learn through trial and feedback. DeepMind presents AdA as evidence that reinforcement learning can support this kind of flexible, in-context adaptation.
From broad training to learning on the task
Foundation models are large AI models trained on broad datasets so they can perform tasks beyond the ones they were explicitly trained to do. The source article describes large language models such as GPT-3 as one example: they predict text tokens and can respond to different tasks through prompts and a few examples.
Earlier foundation models relied mainly on self-supervised training. AdA takes a different route. It is a reinforcement learning agent trained across many tasks, with the aim of building skills that can transfer when it encounters a new challenge.
Rather than choosing tasks at random, the training process selects challenges just above AdA’s current ability. Those tasks call for experimentation, navigation, coordination and, at times, dividing work with other agents. This approach keeps the agent facing problems that stretch its existing skills.
Training for a changing set of challenges
AdA was trained through numerous runs in XLand, a 3D environment. The article describes training across millions of runs in XLand as the process that produces the reinforcement learning model. The environment provides a setting where the agent has to respond to different tasks rather than repeat one fixed routine.
The model uses a custom Transformer architecture. DeepMind says it can store more information, which supports efficient training. The team also uses a teacher-student distillation approach to accelerate learning and train larger models.
The paper reports a model with 265 million parameters and says the method also showed that 500 million parameters are possible. These figures describe the models discussed in the research, rather than a measure of how well AdA performs on every task.
Adapting without changing network weights
When AdA encounters a task it has not been trained on, it explores in a structured way. DeepMind describes the behavior as “hypothesis-driven exploration”: the agent uses what it learns from its interactions to refine its strategy and move toward optimal performance.
The adaptation takes place in the model’s context window. According to the article, AdA can learn a difficult task in a few minutes, and it does so without updating the network’s weights during that process. In this sense, its few-shot ability resembles the way a language model can use a small number of examples in context.
DeepMind says the time needed is similar to that of human players. The paper describes the agent as adapting across a broad, open-ended task space on a timescale similar to human players, based on only a few interactions with a task. That claim is about the tasks and setting studied in the work.
What the result may mean for reinforcement learning
AdA is based on black-box meta-reinforcement learning. DeepMind argues that the approach can scale, challenging earlier assumptions about the method. The result suggests that an agent can learn a general way to adapt through extensive training, then apply that ability to new tasks without changing its weights each time.
The research points toward a possible role for reinforcement learning models in future real-world problems. The article frames this as a prospect: models like AdA could become a foundation for useful reinforcement learning systems, drawing on the scaling seen in language models and other foundation models.
For now, the reported evidence centers on AdA’s performance in XLand and its ability to adapt to held-out tasks there. The central idea is that broad reinforcement learning training can prepare an agent to investigate a new challenge, use a small number of interactions to improve its strategy, and do so within the model’s context.