Runway has made Praxis-1, its model for controlling robots, available with open weights and source code under the Apache 2.0 license. The system applies what it learned about movement in video to robot control, using camera images, text instructions, and position information to choose movements as new images arrive.
From watching movement to choosing robot actions
Praxis-1 starts from a video model trained to learn how objects move and interact in ordinary footage. Runway then fine-tuned that model using videos of human actions and robot data. The company describes it as a World Action Model: it predicts changes in a scene and uses those predictions to determine corresponding robot movements.
In use, the model takes in a view of the environment, an instruction, and information about the robot’s position. It estimates what may happen next, calculates a movement, then adjusts its control as more camera frames arrive. This connects visual understanding directly to action rather than treating the image and movement as separate problems.
Runway says this video-first approach could reduce the need for costly robot-specific training collections. However, Praxis-1 still requires specialized robot data when it is adapted to a new machine, so the amount of data saved compared with systems without general video pretraining remains an open question.
Early tests show results across several robots
Since late September, robotics companies Standard Bots, Noble Machines, and Ultra have tested Praxis-1 on different systems. Those include a humanoid robot, an industrial robot arm, and a dual-arm warehouse robot. The initial release was limited to select partners; the model and code are now publicly available under the stated license.
Runway’s research report describes adaptations using substantial robot recordings. For a Franka robot arm, the company used about 277 hours of recordings from the public DROID research dataset, including examples of robots grasping and moving objects. For a dual-arm robot, it used between 1,400 and 2,600 hours of training material.
On Nvidia’s RoboLab-120 leaderboard, which evaluates sorting, grasping, and stacking across 120 simulated tasks, Praxis-1 recorded a 46.2 percent success rate. FLUX 3 Action, another World Action Model, scored 42.9 percent in the same conditions. Praxis-1 ranked first in that model category and fifth overall, behind two AI agents and two robot models with different architectures. Runway conducted these tests, and the results have not been independently verified.
Runway also compared the models on a real Franka arm after fine-tuning with DROID data. Across ten tasks, each attempted ten times, Praxis-1 completed 88 attempts successfully; FLUX 3 Action completed 80. Tasks included placing a banana on a plate and closing a drawer.
Adapting to a robot it had not seen
A separate experiment examined whether Praxis-1 could transfer to a robot type absent from its earlier training. Runway used recordings from a mobile robot platform with a gripper arm that Google had previously used to develop RT-1. After fine-tuning with about 95 hours of data from that robot, Praxis-1 achieved an average 80 percent success rate in simulated tests, according to Runway. Google’s original RT-1 scored 52.4 percent on the same benchmark.
The simulated tasks covered four types of activity, including grasping soda cans, opening and closing drawers, and placing an apple in a drawer. The results suggest the model can be adapted beyond the robot systems represented in its prior training, though the experiment still involved additional robot-specific data.
Useful control, with practical limits
Praxis-1 can mistake objects that look alike and has trouble with vague instructions. It also cannot reliably plan a complex sequence by itself. For longer jobs, people need to spell out the individual steps, which makes the model better suited to carrying out defined movements than deciding how to complete an entire task.
Runway expects the system could eventually work alongside general-purpose AI models that break a broader request into smaller actions. Under that arrangement, one system could organize the task while Praxis-1 handles the robot’s rapid movements. That is a future possibility described by Runway, rather than a capability established by the reported tests.
Developers can access the model through Hugging Face after signing up and accepting the access terms, with source code on GitHub. Runway says pre-adapted versions for different robots are available, while the base model needs further training for new systems. The standard version requires a GPU with 80 GB of memory; a compressed version is estimated to fit on an RTX 5090 with 32 GB of VRAM, although peak memory use has not yet been measured.