Nvidia just unveiled Cosmos 3, an open AI world model built to help robots, self-driving cars, and other physical systems understand and predict real environments.
Why This Matters
Nvidia keeps pushing past chips into full AI models and software. This positions the company as a key platform for physical AI.
What Sets It Apart
Trained on 20 trillion tokens of multimodal data, nearly a billion images, 400 million real and synthetic videos, plus audio, text, and action data from humans and robots.
The action data is the game-changer. “That action data is what makes Cosmos different from a regular video generator,” said Ming-Yu Liu, VP of Nvidia's Cosmos Lab. It models how machines actually move.
Developers can simulate actions in physical spaces and build custom models on top for robots and vehicles. Cosmos 3 generates details like robot joint angles, gripper positions, and trajectories.
Open and Collaborative
Like Nvidia's early Nemotron models, Cosmos 3 is open. This lets hardware makers customize it easily. Partners include Agile Robots, Black Forest Labs, and Runway. It can safely create rare scenarios like collisions or tricky road events.
Two Versions Now
- Super model for high-accuracy physics tasks.
- Nano model for super-fast results.
An edge model for local running is on the way.
The Bigger Picture
World models are heating up as AI moves into real-world tasks. Nvidia bets the next wave needs to predict, simulate, and act physically.
“Ultimately what a world model wants to achieve is to help physical agents to become more generalizable,” Liu explained.
