Training a robot for a new task often means a person remotely controls it while the system learns. Nvidia’s GR00T 2 is designed to reduce that effort by learning from human movement and translating demonstrations into robot actions.
Predicting what the robot will see next
GR00T 2 takes a different approach from systems that map camera images and language instructions directly to motor commands. Nvidia describes it as a “world-action model”: it predicts both the robot’s next movement and a short video of what the scene should look like afterward.
For example, as a robotic arm reaches to pick up an object, the model predicts how the camera view should change. That visual forecast is intended to help transfer movement sequences between robots with different physical forms.
Nvidia trained the model using more than 400,000 hours of video showing people doing tasks with head-mounted cameras. The training ran on a cluster with more than a thousand GB300 chips.
Turning demonstrations into robot control
To teach a new task, a person demonstrates the steps using tools such as data gloves, specialized hand grippers, or a finger exoskeleton. The researchers combine those recordings with general movement data from the robot that will perform the task.
From there, GR00T 2 predicts the robotic arm in place of the human arm and derives control commands to match. In lab tests, a robot with five-fingered hands lit a match, sliced a banana, and tossed packages onto a conveyor belt without being teleoperated for those specific tasks.
Results depended heavily on the task: reported success rates ranged from 10 to 80 percent. In a computer simulation, the system learned to hammer nails from a single video of a human hand movement. For 50 selected tasks where the model was already performing well after a few examples, less than three minutes of conventional training raised the success rate from 40 to 95 percent.
Planning helps with longer jobs
Tasks involving multiple steps can use a higher-level language model alongside GR00T 2. Nvidia says this model can divide an assembly job into steps, mark grip points in the camera image with colors, and run helper routines if a part falls off. GR00T 2 handles the gripping movements.
In one test, researchers scattered toy-model parts across a table. The combined system assembled them successfully in at least 97 percent of trials. Without the corrective language model and additional fine-tuning, toy car assembly succeeded in 23 out of 100 attempts.
The team also used repeated trial and error on the physical robot to train precise actions such as plugging in network cables or chargers. These results suggest that video-based learning can be combined with other training methods when a task calls for careful handling or a sequence of actions.
Speed and sensing remain challenges
Nvidia developed a smaller variant, GR00T 2 Turbo, for bipedal robots. It processes fewer image details and runs directly on the Jetson Thor onboard computer. In testing, a humanoid robot scooped popcorn, cleared tables, and wiped up liquids with a towel under its foot. The cleanup task worked in just under half of attempts.
The researchers say the system still has important limitations. It does not yet process direct touch-sensor data and can miss small errors, such as screws that are only partly turned. Those gaps matter because a robot may need to recognize not just where an object is, but whether a delicate physical action has actually worked.
Planning can also introduce delays. During trash sorting, the robot was idle for roughly a third of the total time while it waited for the planning model, which took more than seven seconds on average to make its next computation. So while GR00T 2 aims to reduce the effort required to teach robots, reliable operation still depends on improving both feedback and response time.