The race to build robots that can actually understand and navigate the real world just got a massive acceleration. NVIDIA has launched Cosmos 3, a groundbreaking open foundation model it’s calling the world’s first fully open “omnimodel” for physical AI — and the implications for robotics, autonomous vehicles, and industrial automation are enormous.
Unveiled at NVIDIA GTC Taipei on May 31, 2026, Cosmos 3 doesn’t just push the boundaries of what AI can do in a lab. It’s designed to be deployed in the physical world, and it brings a fundamentally new architecture to the task.
What Makes Cosmos 3 Different?
At the heart of Cosmos 3 is a breakthrough mixture-of-transformers architecture that combines three distinct capabilities in a single model: vision reasoning, world simulation, and action prediction. No prior open model has unified all three.
Most AI models today are specialists. A vision-language model (VLM) can look at an image and describe it. A world model can simulate what might happen next in a scene. A robot action policy model can translate goals into motion. Cosmos 3 does all three — natively, from one model, without stitching together separate systems.
The practical result is a model that can understand object interactions and spatial relationships, simulate how a physical environment will evolve, and then generate the action trajectories needed to operate within it. Whether the application is a warehouse robot, an autonomous vehicle navigating city streets, or a vision AI agent monitoring an industrial facility, Cosmos 3 is designed to be the single foundation these systems build on.
The Omnimodel Vision: More Than Multimodal
NVIDIA describes Cosmos 3 as an “omnimodel” — a term the company has coined to describe models that go beyond traditional multimodal AI. Where standard multimodal models handle text and images, an omnimodel like Cosmos 3 natively understands and generates across text, images, video, ambient sound, and physical actions simultaneously, all with physics accuracy.
This matters enormously for physical AI development. Training a robot traditionally requires massive real-world data collection — thousands of hours of robot arm demonstrations, vehicle sensor logs, or industrial camera footage — followed by months of simulation and fine-tuning. With Cosmos 3, NVIDIA says that training and evaluation cycles can be compressed from months to days by generating high-quality synthetic training data that respects real-world physics.
Jensen Huang, NVIDIA’s founder and CEO, put it plainly at the GTC Taipei keynote: “The big bang of physical AI is just around the corner thanks to breakthroughs in multimodal reasoning language, vision and world models. The Cosmos 3 family of open, frontier omnimodels gives developers a generational leap in ability to build robots, autonomous vehicles and vision AI that perceive, reason, plan and act in the physical world.”
Benchmark Performance: A Clear Leap
NVIDIA’s claims are backed up by benchmark results. Among open models, Cosmos 3 currently ranks first across several leading physical AI evaluation frameworks:
- Physics-IQ and PAI-Bench for world generation accuracy
- R-Bench for physical world reasoning
- RoboLab and RoboArena for action policy performance
- VANTAGE-Bench and TAR for vision understanding in autonomous driving contexts
These aren’t niche leaderboards. They represent the core challenges the robotics and AV industries have struggled to solve for years: does the model actually understand physics? Can it generate realistic future states? Can it turn intentions into motion?
Model Variants: From the Cloud to the Edge
NVIDIA is releasing Cosmos 3 in a family of sizes tailored for different deployment environments:
Cosmos 3 Nano — Built around an 8-billion-parameter reasoner paired with an 8-billion-parameter generator, Nano is designed for real-time or near-real-time applications that need fast video and action reasoning. Think edge devices in factories, or onboard computing in autonomous vehicles.
Cosmos 3 Super — The powerhouse variant, pairing two 32-billion-parameter transformers. Super is aimed at post-training workflows for robotics and AV models where the highest physics accuracy and generation quality are required — i.e., the fine-tuning phase before a Nano-class model gets deployed to the field.
Cosmos 3 Edge — Not yet available but announced as coming soon, Edge targets real-time inference at the hardware edge. This is the version that will eventually power onboard physical AI at the device level, without cloud connectivity.
Both Nano and Super are available now on Hugging Face and through NVIDIA’s build.nvidia.com developer platform. They can also be deployed as NVIDIA NIM microservices for enterprise and cloud integration.
The Cosmos Coalition: Building Open World Models Together
Alongside the model launch, NVIDIA announced the NVIDIA Cosmos Coalition — a new global alliance of AI labs and robotics companies committing to advance open world models together. Founding members include Agile Robots, Black Forest Labs, Generalist, LTX, Runway, and Skild AI.
The coalition model is a deliberate counterweight to the increasingly closed ecosystem of frontier AI. By building openly and contributing models, research, and evaluation benchmarks back to the ecosystem, coalition members get access to Cosmos 3 technologies, training tools, and NVIDIA DGX Cloud infrastructure for large-scale training.
It’s a meaningful signal: NVIDIA is betting that physical AI advances faster when the foundational world model layer is open and collaborative, even as the applications built on top remain proprietary.
Who’s Already Building on Cosmos 3?
Early adopters span some of the most demanding physical AI verticals:
- Robotics: Agile Robots, Doosan Robotics, LG Electronics, Samsung Electronics, and Skild AI are leveraging Cosmos 3 for manipulation and mobile robot tasks.
- Autonomous vehicles: Li Auto is using the platform to accelerate AV training pipelines.
- Vision AI and industrial: Milestone Systems, Linker Vision, Fogsphere, and Centific are building vision AI agents for smart spaces, manufacturing, and industrial monitoring.
The breadth of adoption at launch is notable. It suggests that the robotics and AV industries were hungry for exactly this kind of unified foundation — one that bridges the gap between simulation and real-world deployment without requiring massive internal model development.
Why This Matters for the Physical World
The AI industry has largely been a software story for the past several years — language models, code generators, image synthesizers. But the next frontier is unambiguously physical: robots that can operate in unstructured environments, vehicles that can handle edge cases, industrial systems that can adapt.
The core bottleneck has been data and simulation. Building a robot that generalizes beyond its training scenarios requires either enormous amounts of real-world data (expensive, slow, dangerous to collect) or simulation environments that accurately model physics (historically, a massive engineering undertaking requiring specialized tools).
Cosmos 3 attacks both problems simultaneously. Its synthetic data generation capabilities let developers bootstrap training without physical data collection. Its physics accuracy means that models trained in simulation are more likely to transfer successfully to the real world.
That said, the technology still has limits. Cosmos 3 is a foundation model, not a finished product — it requires significant customization and fine-tuning for specific robotic tasks. NVIDIA also acknowledges that Cosmos 3 Edge, the version best suited for fully onboard deployment, hasn’t shipped yet.
But the trajectory is clear. The company that has been the infrastructure provider for AI’s software revolution is now laying the same kind of open foundational infrastructure for the physical AI era. For anyone building robots, autonomous systems, or industrial vision agents, Cosmos 3 is the most significant open release of 2026 — possibly since transformers reshaped NLP.
The big bang of physical AI, Jensen Huang promised, is just around the corner. With Cosmos 3, he’s handed developers the match.
NVIDIA Cosmos 3 Super and Nano are available now on Hugging Face and build.nvidia.com. Cosmos 3 Edge is coming soon.
