Robot training videos need more than a summary. They need labels that show what actions occur, how a robot performs along the way, and whether it completes a task. Nvidia’s Flora is an open system designed to provide all three, with a projected labeling cost far below that of GPT-6 Astra.
Three kinds of labels for robot videos
Video annotations help developers choose useful training examples, teach robots through trial and error, and monitor performance during deployment. Nvidia’s GEAR research lab says existing tools often provide only one of these label types, while large language models can produce several but may be expensive at the scale robotics requires.
Flora combines action descriptions with performance and outcome judgments. It is already being used in training Nvidia’s GR00T models, the company says. Nvidia reports that the system can annotate up to 200,000 hours of video per day across 512 GB300 GPUs.
How Flora breaks down a video
The system consists of three fine-tuned Qwen3.5 models. A 9B model identifies segments where actions take place. One 27B model describes those actions while taking surrounding events into account.
The other 27B model, Flora-Rewarder, evaluates robot performance in several ways. Depending on the prompt, it can rate an episode’s success from 1 to 5, classify each half-second step as normal, suboptimal, or failure, or score performance using self-defined rubrics.
Those outputs serve different purposes. Action segments help locate events in a video, while step-level judgments can expose a mistake that occurs before the end of a task. Episode ratings offer a broader view of whether the overall attempt succeeded.
The cost estimate comes with a caveat
Nvidia estimates that Flora costs about 72 cents to label an hour of video, compared with roughly $88.50 for GPT-6 Astra. In a test involving 50 videos, the company says Flora scored slightly better, even though Astra could use its full reasoning capacity. Claude Opus 5 performed much worse in that test and cost about $42 per hour.
The Flora price is a projection rather than a production measurement. Nvidia calculated it using an assumed rental price of $10 per GPU per hour, so the estimate depends on that cost assumption.
Benchmark results also vary by task. On public WGO-Bench, Flora is strongest at identifying where actions begin and end, but trails Astra in the overall score and Gemini 3.8 Flash in description accuracy. Nvidia’s researchers attribute Flora’s gains mainly to temporal localization.
On Nvidia’s DenseCaptionBench, Flora leads across metrics for videos of people, but ranks behind Astra on robot videos. The researchers also acknowledge a limitation in the evaluation: GPT-6 Astra judges descriptions while also being one of the models being tested.
Better at some errors, with tradeoffs
Flora’s results on RoboRewardBench point to a particular advantage: recognizing partial completion. When a robot tips a bottle over without knocking it off a table, Flora assigns 3 out of 5 points, matching the human reference, while Astra gives one point. Overall, Flora’s error rate is 27.8 percent lower than Astra’s on this benchmark.
Flora-Rewarder catches 53.3 percent of failed steps within an episode. No competitor exceeds 11.7 percent, and Astra does not catch any in the reported comparison. But Flora also falsely marks about 11 percent of normal steps as errors. Astra remains ahead in rubric-based scoring and trajectory ranking.
Action detection can be notably strong in busy footage: in a video with many short hand movements, Flora identifies 34 of 37 actions, while Astra identifies four. Yet Flora may split a longer action into too many pieces or describe it incorrectly. In one example, it calls a robot hand wiping a plate with a cloth “folding a napkin.”
An open tool within Nvidia’s robotics push
Nvidia presents Flora as an open alternative to commercial services such as Scale AI’s Physical AI Data Engine. Researchers can use it to process robotics datasets and explore approaches such as transferring human demonstrations to robots. The models are available on Hugging Face.
Flora adds video labels and reward signals to Nvidia’s wider effort to prepare robotics training material through simulation and automated processing. At GTC earlier this year, Nvidia showed a pipeline that prepares, multiplies, and scores training data. Flora extends that work with more detailed labels from video.
The project also fits Nvidia’s broader strategy across robotics hardware, data, simulation, and AI models. Google’s recently announced Gemini 4 Argon targets a similar training task, identifying work steps in videos and flagging failed actions. For developers, Flora’s appeal will depend on whether its lower estimated cost and error detection outweigh its occasional labeling mistakes.