What Is Nvidia Cosmos?

Nvidia Cosmos is a platform built to accelerate the development of “physical AI,” the artificial intelligence powering anything robotic. Developed by chip manufacturer Nvidia, Cosmos includes a video data curation pipeline, various tools for customizing data to specific use cases and — perhaps most importantly — a family of specialized AI models called “world foundation models.”

Training physical AI systems is incredibly labor-intensive, often requiring the collection, labeling and categorization of millions of hours of real-world footage. Cosmos’ world foundation models (WFMs) aim to tackle this issue by generating three-dimensional simulations of the physical world to train robots — whether it’s a factory robot assembling a product, a humanoid robot interacting with humans or an autonomous vehicle navigating the road. In other words: WFMs make synthetic training data so developers don’t have to rely entirely on costly real-world data collection and processing.

Nvidia unveiled Cosmos at the 2025 CES conference in Las Vegas, alongside several other product launches and upgrades. Among them is a new feature on its Isaac simulation platform, enabling robot builders to generate large volumes of synthetic training data from just a few examples. The company also upgraded its Omniverse platform with a “Mega” operating system that lets developers create, test and optimize their robot fleets in a digital twin environment before deploying them in the real-world. 

Together, Nvidia hopes these products will expand what is possible in physical AI — an industry projected to be worth $45 billion by 2029.

“Everything that moves — from cars and trucks to factories and warehouses — will be robotic and embodied by AI,” Jensen Huang, Nvidia’s founder and CEO, said in a statement. “Nvidia’s Omniverse digital twin operating system and Cosmos physical AI serve as the foundational libraries for digitalizing the world’s physical industries.”

What Is Nvidia Cosmos?

Nvidia Cosmos offers a suite of foundation models that can generate “controllable, high-quality” synthetic data to train robots, driverless cars and more, according to Nvidia. The models are divided into three categories:

1. Nano, for real-time, low-latency inference and edge deployment.
2. Super, a “highly performant baseline” model for “out-of-the box” fine-tuning and deployment.
3. Ultra, for maximum accuracy and quality outputs; provides best fidelity knowledge transfer for distilling custom models.

The size of the models range from 4 billion to 14 billion parameters, with Nano being the smallest and Ultra being the largest. Essentially, parameters correspond to an AI model’s problem-solving skills, so models with more parameters generally perform better than those with fewer parameters.

In addition to the WFMs, Cosmos includes a 12-billion-parameter “unsampling” model for refining text prompts, a 7-billion-parameter video decoder optimized for augmented reality and a guardrail model to ensure “responsible” use. It also includes fine-tuned models for applications like generating multisensor views for autonomous vehicles. 

Nvidia says all of its Cosmos models were trained on 9,000 trillion tokens (bits of raw data — in this case, video footage) from 20 million hours of real-world environmental, industrial, robotics, and driving data, as well as human activities like walking and manipulating objects. Users can also fine-tune the models with their own data.

Nvidia is joining a growing roster of companies building tools to simplify the integration of artificial intelligence into robots. Other top players include cloud provider AWS, game development company Unity and Genesis, an open source computer simulation system created by a group of university and private industry researchers (including from Nvidia) that can reportedly train robots 430,000 times faster than reality. 

Nvidia Cosmos is now available for free on both Hugging Face and Nvidia’s own NGC catalog under an open model license, meaning developers can freely access and use all of the models, so long as they have Nvidia hardware. Several companies have begun using the platform, according to Nvidia, including humanoid robot makers Galbot, Agility Robotics and Figure AI, as well as self-driving car companies Uber, Wayve and Waabi.

More on NvidiaWhat Is Nvidia’s Chat with RTX?

What Can Nvidia Cosmos Do?

Here are some of the key features and capabilities of Nvidia Cosmos.

Video Search

Nvidia says Cosmos can simplify the video tagging and search process, helping to make robots that are ready for the real world. So, whether it’s footage of a snowy road in front of a self-driving car or a busy warehouse, the platform is able to understand spatial and temporal patterns, saving developers time and costs associated with training data preparation.

‘3D-to-Real’ Synthetic Data Creation

With Cosmos, developers can transform their 3D simulation data into “photoreal” synthetic video, according to Nvidia. By pairing the platform with Omniverse, users can create 3D environments that represent their training needs. Then, they can generate photorealistic videos that are “precisely controlled” by the 3D scenes, creating “highly tailored” synthetic datasets.

Policy Model Training and Evaluation

Cosmos’ world foundation models — when fine-tuned for “action conditioned video prediction” — enable scalable and repeatable training and evaluation of policy models, according to Nvidia, which define strategies for physical AI systems by mapping states to actions. The company says developers can use these models instead of relying on “risky real-world tests” or complex simulations of tasks like obstacle navigation and object manipulation, which can help optimize machines’ performance and improve their reliability. 

Predictive Intelligence

Cosmos is designed to bring advanced predictive intelligence — or “foresight,” as Nvidia puts it — to physical AI, enabling systems to anticipate future scenarios and make decisions based on those predictions. The platform can do this by generating predictive videos based on past data and text prompts, enabling robots and other automated machines to navigate and adapt to dynamic physical environments safely and efficiently.

‘Multiverse’ Simulation

When paired with Omniverse, Cosmos can help developers simulate multiple outcomes of real-time scenarios, accelerating the decision making process for robots, autonomous vehicles and other artificially intelligent machines. Creating a sort of “multiverse,” Nvidia says Cosmos and Omniverse can work together to explore “all possible future outcomes,” ultimately selecting the best path for “enhanced precision and reliability in complex environments.”

Related ReadingThe Future of Robots and Robotics

How Does Nvidia Cosmos Work?

Nvidia Cosmos’ world foundation models work similarly to large language models (LLMs), insofar as both process vast amounts of data and produce complex outputs based on the patterns they learned from that data. 

However, while LLMs are trained on text data (books, articles, social media posts, etc.) to perform tasks related to natural language processing and generation, WFMs are designed to make photorealistic, “physics-aware” simulations for training robots in real-world applications. Using data from text, images, video and motion sensors, these models render detailed worlds, and then identify and label the physical attributes within those worlds — maintaining accurate spatial awareness, physics and object permanence throughout.

The focus isn’t just on generating AI content, but on teaching AI to understand the physical world well enough to operate in it effectively. Cosmos can, for example, make video footage of boxes falling from shelves, which can be used to train a robot to recognize accidents. Or it can simulate a self-diving car encountering an animal on the side of the road, teaching the vehicle to anticipate the animal’s movements and slow down if necessary. Meanwhile, all of this training data is synthetic, making it more accessible to a broader range of developers.

“The ChatGPT moment for robotics is coming. Like large language models, world foundation models are fundamental to advancing robot and [autonomous vehicle] development, yet not all developers have the expertise and resources to train their own,” Huang said in a statement. “We created Cosmos to democratize physical AI and put general robotics in reach of every developer.”

Frequently Asked Questions