World Labs is pushing a direct answer to one of robotics' hardest practical problems: robots need far more experience than most labs can afford to collect in the physical world. Its new Real-to-Sim-to-Real engine, or R2S2R, turns recorded robot tasks into interactive simulations where control models can train and be evaluated before they ever touch real hardware.
The company, founded in 2024 by AI researcher Fei-Fei Li, frames the system as the first concrete robotics application of its broader work on world models. The pitch is straightforward: if robot intelligence depends on experience, then developers need a way to scale the worlds where that experience happens.
Why simulation matters for robot learning
World Labs argues that the main barrier to robot deployment is not the shape of the model architecture. The harder constraint is the amount and variety of experience a robot needs before it can act reliably around real objects, changing conditions, and edge cases.
Physical robot data is costly to gather and difficult to control. Online video can show many scenes, but it does not systematically cover the full spread of object arrangements, physical conditions, and failure states that a robot may encounter.
R2S2R is designed to close that gap by rebuilding a real task as a virtual environment that is not only visually similar, but physically useful. The engine captures the robot, sensors, environment, and task demonstrations, then recreates them as a simulation where developers can change conditions and observe how a control model responds.
One task becomes thousands of variations
The core idea is that a single real-world task can become thousands of controlled training cases. World Labs says the engine can vary lighting, camera angle, object position, object count, the surrounding environment, and physical properties such as friction.
That matters because real robot failures often come from small changes. A cable sits at a different angle. A camera sees the scene from another viewpoint. A surface behaves differently. A model that only learned one narrow version of a task may look capable in a demo but break when conditions shift.
World Labs checks the fidelity of its simulations by running the same action sequence in both simulation and reality. It compares what the robot observes, how objects move, and whether the outcome matches.
The examples shown cover several types of manipulation, including rigid, movable, and deformable objects. The tasks include cable routing, inserting an elastic cable end into a hole, and packing a box with both hands.
Models trained in simulation move to real robots
World Labs says its control models train in simulation and then transfer to physical robots. One test platform was ALOHA, an open-source dual-arm design from Stanford that is operated through puppeteering with two smaller control arms.
The article describes ALOHA as a common reference platform in robotics research because its blueprints are public and its setup costs a fraction of commercial systems. That makes it a useful proving ground for approaches that need to be tested by other research teams.
According to World Labs, the models each ran for one hour across four additional robot platforms without human intervention. The tested tasks included wrapping a power cord around a refrigerator with both hands, precisely repositioning test tubes, and separating thin objects such as markers or pencils from a dense jumble.
The company also says the system is not tied to one specific control model or robot type. Once a world has been reconstructed, it can later be reused for new models and different robots.
Evaluation may be as important as training
Robot development has a testing problem as well as a training problem. World Labs argues that progress has lagged behind language models partly because evaluating control models has usually required real hardware tests.
The company does not claim that a useful simulator must perfectly match real-world success rates. Instead, the important question is whether it produces the same development signals: where a model fails, which version is better, and whether improvements in simulation carry over to the physical robot.
World Labs tested this idea with a two-handed cube handoff between the arms of an ALOHA robot. The simulation reproduced borderline cases where the robot barely grasped the cube by its edge, along with matching failed attempts.
Across model types including GR00T N1.6 and π₀.₅, and across training stages, model rankings in simulation and reality stayed largely the same. That pattern held for both known cube positions and previously unseen ones. Each checkpoint was evaluated using 2,000 simulated and 100 real runs.
If that ranking behavior holds more broadly, development teams could use simulation to discard weak model versions earlier. Real hardware tests could then be reserved for the most promising candidates.
World models move toward physical action
World Labs connects R2S2R to its taxonomy of world models. In that framework, simulation is central because it turns a world into a place where software agents can act, learn, and be tested.
The company draws a comparison with autonomous driving, where some successful Level 3 and Level 4 systems train on a mix of real and simulated data. Its long-term claim is that scaling robot intelligence requires scaling the worlds in which robots learn.
There are still open questions. The source notes that it remains unclear how well these results will transfer to more complex environments, other robot types, and less controlled everyday situations.
World Labs has been building toward this direction since its founding. An early system generated walkable 3D environments from single photos. More recently, the company raised one billion dollars in venture capital to extend its world models into robotics and science.
The work also sits inside a wider debate over what world models should mean in robotics. An international research team recently tried to define the term more uniformly, distinguishing world models from pure video generators. Related work includes World Action Models, which connect near-future predictions directly to control commands, and Orca, a method from China that lets a robot learn tasks by watching video without real motion data during training.
For World Labs, R2S2R keeps simulation and policy separate. The simulator provides the world; the control model learns inside it. The result is a robotics strategy built around generating enough structured experience for machines that must eventually act in the physical world.