Robotics research is exploring whether methods trained across many tasks and machines can help robots adapt to new situations. Google DeepMind’s work combines data from research labs with AI approaches that may help robots interpret instructions, plan actions and learn from experience.
Learning across different robots
Google DeepMind introduced Open X-Embodiment, a robotics dataset developed with 33 research institutes. At the time of the announcement, it included 500+ skills and 150,000 tasks gathered from 22 robot embodiments. The researchers described the effort as a shared foundation for training models that can control different kinds of robots and generalize across instructions and tasks.
DeepMind trained its RT-1-X model on the dataset and used it to train robots in other labs. The reported success rate was 50% compared with the in-house methods those teams had developed. That result offers an early sign that sharing data and training approaches may help researchers move beyond systems designed for only one specific robot or task.
Vincent Vanhoucke, Google DeepMind’s head of robotics, says the team is focused on general-purpose methods rather than a single machine expected to do everything. A method that works across industrial, home or sidewalk robots could be adapted to a particular machine and its job. The robot itself can remain specialized while the underlying AI techniques become more broadly useful.
Why perception changed the outlook
Vanhoucke traces the growing interest in robotics to progress in perception. As computer vision and audio processing improved, researchers began to see more clearly how machines might operate in everyday environments. Robots need to recognize what is around them before they can make sense of a task or act on it.
Google Research and DeepMind efforts eventually merged, and Vanhoucke’s background included computer vision, speech recognition and general AI. He described the core challenge as intelligence: helping robots perceive, understand and control what they do in the physical world.
Work with the Everyday Robots team helped bring that idea into practice. The teams had collaborated for years, including an experiment in which researchers put discontinued robot arms together and trained them to grasp objects. The arms received a reward when they succeeded and a negative signal when they failed. That trial demonstrated how machine learning and perception could be used to teach robots a generalized grasping skill.
A portion of the Everyday Robots team joined DeepMind’s robotics effort, and the team kept using the robots. Vanhoucke said the work continues, with a stronger emphasis on the intelligence behind robot behavior than on building the machines themselves.
Common sense and language models
Generative AI could play a central role because language models encode information about the everyday world, not just language. Vanhoucke pointed to simple expectations, such as finding a coffee cup in a kitchen cupboard or on a table, and knowing that placing a table on top of a cup makes little sense.
People rarely have to spell out this kind of common sense, but it is difficult to encode directly into an embodied system. If a robot can draw on such knowledge, it may be better equipped to plan tasks, manipulate objects and interact with people. The model’s reasoning can also be combined with perception and used in a simulated environment.
This opens a path for AI systems to help robots make sense of instructions and possible actions. The challenge is turning general knowledge into reliable behavior: a robot must connect what it understands to the objects and conditions it actually encounters.
Using simulation to plan ahead
Simulation can help researchers gather experience and explore what might happen when a robot acts. But a simulated environment is only an approximation of reality. Its physics and visual rendering need to reflect the real world closely enough for lessons learned in simulation to be useful outside it.
Generative AI may offer another way to explore possible outcomes. Instead of relying only on a physics simulator, a model could generate what a scene or future state might look like after a robot takes an action. The robot could use those imagined outcomes to check whether an action is likely to achieve its goal and plan what to do next.
That approach resembles a robot rehearsing before acting. It would let the system consider possible futures without having to test every choice in the physical world. The idea remains part of an evolving research effort, alongside shared datasets, perception and general-purpose methods.
For now, specialized robots remain the more established approach. Vanhoucke sees signs that methods spanning different machines are becoming plausible, but whether broad-purpose robotics will succeed as a technology and business model is still an open question. The work aims to establish the technical foundation that could make wider applications possible.