Nvidia Unveils Flora: A New Tool for Training Robots

Training robots is a complex and expensive process that requires the analysis of massive volumes of video footage. To address this, a team of researchers at Nvidia, known as GEAR, has created a new open-source platform called Flora. The system aims to automate video labelling, allowing robots to learn from human actions more effectively.

How Flora Works

Traditionally, preparing robots requires videos containing three types of data: action descriptions, progress assessments, and task completion status. Existing tools previously handled these functions separately, while powerful models like GPT-6 Astra were too expensive for large-scale projects.

Flora consists of three optimised Qwen3.5 models. The 9B model segments the video, while the 27B models provide detailed descriptions. Particularly significant is the Flora-Rewarder, which evaluates the success of a robot’s actions on a scale of 1 to 5, determining whether each step was correct. According to Nvidia, processing one hour of video using Flora costs approximately 72 cents, whereas the same task costs $88 using GPT-6 Astra.

Efficiency and Challenges

During testing, Flora demonstrated a high proficiency in temporal action localisation. For instance, in scenarios where a robot knocks over a bottle—a task where other models often fail—Flora was able to assess the situation accurately and assign precise scores. However, the system still has limitations; it sometimes breaks actions into overly granular segments or misinterprets context, such as mistaking the cleaning of a plate for folding a napkin.

Nvidia’s goal is to provide robotics researchers with a low-cost, effective alternative to commercial services. This move aligns with the company’s broader strategy to control the entire robotics ecosystem, from hardware to AI models.

What This Means for the Future of Robotics

Nvidia’s decision to release Flora as open-source is a significant signal for developers. The company is actively working to eliminate the data deficit required for robot training. While competitors like Google’s Gemini 4 Argon are attempting to implement similar functionality, Nvidia’s approach focuses on scalability and cost reduction.

For researchers and startups working in robotics or computer vision, the availability of such open tools means that testing and developing complex algorithms will be possible on much smaller budgets. Technologies like Flora are accelerating the timeline for when robots will become more flexible and independent in our daily lives.