A virtual duck learned to walk around a room without falling over and look for a Bengal cat. The experiment points to a changing role for AI in robotics: models can help create simulations, write control code and assess training runs, not just answer questions or generate software.
From a virtual duck to a possible cat helper
The Microduck is a small robot developed by Pollen Robotics and Hugging Face. In a simulation, it was taught to move around and identify Bengal cats nearby. The planned next step is to use a physical version to dispense treats to Leona, the writer’s pet, perhaps by tipping its head so a treat falls from a 3D-printed storage chamber.
That physical robot had not yet arrived, and the simulated duck’s vision system was not good enough for a real product. The cat-butler idea is therefore a work in progress, not a finished device. Still, the simulation made it possible to explore how the robot might behave before trying those actions in the physical world.
AI helps build and improve simulations
The duck was coded and trained in a few hours, a task that the writer says might previously have taken weeks. GPT-6 Sol worked with Nvidia’s Isaac Sim to give instructions to a control model, review videos of the robot’s attempts and suggest changes to training.
That kind of assistance matters because simulations need to reflect the physical world closely. Robot behavior trained in a virtual environment must transfer reliably to the real one. A simulation has to represent details such as camera position, lighting, background and materials, as well as the robot’s movements.
Researchers at the University of California, Berkeley explored whether AI could speed up that work. After seeing Astra render 3D images, Ken Goldberg and his students tried using it with Nvidia’s Newton tool. A lab member showed Astra a video of a real robot and asked it to recreate the scene as a simulation. Goldberg said the system produced a full 3D simulation and set its parameters within an hour.
Astra can also run experiments and review their results, helping automate some of the repetitive work involved in simulation-based robot training. In the Microduck project, the model could inspect how the virtual robot performed and recommend adjustments rather than only generating code.
Learning physical skills takes trial and error
The system got the duck walking without toppling over, but not every experiment worked. An attempt to teach it a moonwalk from a GIF of Michael Jackson ended in a backward jog after hours of effort. The contrast shows both the promise of AI-assisted training and its limits: a model can iterate on a movement, but that does not guarantee it can reproduce the intended skill.
Jim Fan, an Nvidia researcher who has collaborated with Goldberg, describes robot control as similar to the way software agents use tools. Language models can call on visual language action models, or VLA models, that are trained to control physical actions.
Fan, Goldberg and others also open sourced Aspire, an agentic framework that lets robots discover and use new skills. One example involved GPT-6 Astra using a pair of robot arms to solve a Rubik’s cube. Fan said it began with the wrong angle, then reasoned and learned skills over time until it completed the puzzle.
Promise alongside practical limits
At the Conference on Robot Learning (CoRL) in Austin, Texas, Fan demonstrated Astra controlling a robot arm to paint the Golden Gate Bridge using nearby paint and brushes. The example suggests how a model might connect a request with tools and physical actions, and then carry out a multi-step task.
But demonstrations do not remove the engineering challenge. Robots still need to work reliably and efficiently, and that requires significant expertise. In the Microduck experiment, the vision system remained a clear obstacle to deployment.
Even with those limits, faster ways to train robots could matter in manufacturing and e-commerce. Goldberg said industrial robotics and automation firms have shown interest in the approach. If models make it easier for hobbyists and engineers to program and train machines, experimentation could spread beyond specialist labs. For now, the virtual duck offers a glimpse of that possibility, while its real-world cat-treat mission remains unfinished.