NVIDIA Cosmos 3 Nano adds four interactive simulation checkpoints

NVIDIA has expanded its Cosmos world models platform with four new open-weight checkpoints designed for interactive simulation in robotics and autonomous driving. The company announced these additions on November 5, 2026, as part of its Cosmos 3 model family.

Each of the four new checkpoints is based on Cosmos 3 Nano and includes 16 billion trainable parameters. They address different simulation requirements across robotic manipulation and autonomous vehicle development. The release aims to provide a shared foundation for researchers and developers building simulated environments where AI agents can interact, take actions, and observe consequences.

What changed

NVIDIA introduced four specialized variants of Cosmos 3 Nano, each designed for a specific simulation task. Cosmos3-Nano-Transfer-Auto generates synthetic multi-view driving data with support for up to 11 synchronized camera views, including fisheye lenses, plus joint RGB and LiDAR generation. Developers can specify scene structure and traffic motion through per-camera World Scenario Map controls, while text prompts guide visual appearance.

Cosmos3-Nano-Sim-Auto uses causal distillation to stream synchronized camera and LiDAR observations in a few denoising steps. It generates successive chunks of video data and LiDAR information using generated history and updated driving controls, enabling fast streaming and long interactive rollouts.

Cosmos3-Nano-Sim-Transfer accepts spatial controls from a physics simulator and a text prompt to render RGB observations. The system maintains physics and ground-truth state within the simulator while using the model to generate color observations that policies can consume.

Cosmos3-Nano-Sim-Bimanual predicts video sequences from robot or camera actions, supporting interactive two-arm manipulation for specific robot embodiments. It accepts mixed-action inputs defined by a 64-value schema covering coordinate frames, rotations, gripper calibration, and control domains.

All four checkpoints use OpenMDW1.1 licensing, which permits both commercial and non-commercial use. The models are available for download with technical documentation, inference code, and notebooks. NVIDIA provides post-training recipes to help teams customize Transfer-Auto for their specific sensor configurations and driving domains, with the option to then distill the adapted model into Sim-Auto for fast streaming inference.

Why it matters

These checkpoints address core challenges in robotics and autonomous driving. For robotics, Sim-Bimanual predicts visual outcomes from manipulation actions, while Sim-Transfer renders observations from a physics simulator’s spatial state. Both approaches enable agents to explore how actions produce visual consequences across changing scenes and viewpoints.

For autonomous driving, synchronized camera and LiDAR generation solves a fundamental problem: visual appearance and 3D geometry must remain geometrically consistent. Researchers can vary road layouts, lighting, weather, and traffic signal states while studying how driving policies respond to these changes.

The streaming generation capability matters because each observation can inform the next action. By generating predictions sequentially and chaining them together in real time, researchers can explore how actions and observations unfold interactively. This differs from batch prediction by allowing tight feedback loops between the agent’s decisions and the simulated environment.

The open-weight approach significantly lowers adoption barriers. Rather than depending on proprietary APIs with quota limitations or vendor lock-in, developers can download and adapt these models to their own sensor configurations and environments. NVIDIA provides post-training recipes supporting all four training stages, allowing teams to customize models starting from Transfer-Auto and progressively distill them for their specific needs.

However, NVIDIA acknowledges that important work remains. Measuring where these models faithfully support policy learning and evaluation, understanding their failure modes, and improving performance in edge cases are ongoing challenges that the research community must address.

What to test

Before committing these checkpoints to production workflows, verify several critical behaviors:

• Multi-modal alignment: Test whether synchronized camera and LiDAR output maintain geometric consistency across varied scenarios, weather conditions, and camera viewpoints. While the architecture supports joint generation, real-world sensor configurations may introduce drift over time.

• Prediction error accumulation: Evaluate whether prediction quality degrades significantly across long interactive rollouts. The training pipeline includes self-forcing to expose the model to its own mispredictions, but test how this scales beyond demonstration scenarios.

• Adaptation requirements: Verify the computational cost, data volume, and time needed to customize Transfer-Auto to your specific sensor configuration and driving domain. The company provides recipes, but actual results depend on domain similarity and available training data.

• Policy transfer success: Measure whether policies trained or evaluated on generated worlds transfer to real environments or higher-fidelity simulators. Sim-to-real gaps remain a known challenge in robotics and autonomous systems.

• Control schema compliance: Confirm that your action representations match the expected control formats, especially for Sim-Bimanual with its 64-value mixed-action input. Misalignment will degrade output quality.

• Hardware compatibility: The company states the setup was tested on one GB200 GPU but does not specify minimum memory requirements. Test inference and adaptation on your actual hardware before committing to a workflow.

• Temporal consistency: For Sim-Bimanual, verify that timing alignment holds across sequences, ensuring that T action transitions produce T+1 observations without frame drift.

Common questions

What are NVIDIA Cosmos world models?

NVIDIA Cosmos world models are foundation models designed to understand and simulate how physical environments evolve. They combine physical reasoning, world generation, and action generation within a single architecture. The models process and generate multiple modalities including language, images, video, and audio.

Is NVIDIA Cosmos free to use?

Yes, all four released checkpoints use OpenMDW1.1 licensing, which permits both commercial and non-commercial use. The model weights are available for download, and NVIDIA provides open-source inference code, notebooks, and post-training recipes to help developers adapt the models.

What is the difference between a world model and a foundation model?

A world model simulates how an environment changes over time in response to actions and conditions. A foundation model is a large pre-trained network that learns general patterns from broad data. World models typically specialize in capturing cause-and-effect relationships in specific domains, while foundation models aim for broad applicability. Cosmos bridges this by using foundation-scale training but organizing the architecture and post-training recipes specifically for world simulation tasks.

The conclusion

NVIDIA’s release of four Cosmos 3 Nano checkpoints represents a step toward open infrastructure for interactive world simulation. These models move the design conversation beyond static image or text generation toward environments where agents can take actions, observe consequences, and learn from the results. The streaming generation capability and multi-modal support address real constraints in both robotics and autonomous driving.

The open-weight approach removes at least one barrier to adoption. Developers no longer depend on proprietary API quotas or vendor lock-in. However, the practical value depends on how faithfully these models support policy learning in your specific domain. NVIDIA itself signals that measuring this fidelity and closing performance gaps remain work ahead.

Watch for follow-up work addressing frontier tasks: longer rollouts without compounding error, lower computational cost for streaming inference, and validation results showing successful policy transfer from simulated to real environments. Also monitor how quickly the research community contributes domain-specific adaptations and improvements through the open Cosmos framework.

Related reading

• The Recipe That Fed a Waiting World
• NVIDIA Auto Ontology: Open semantic layer for AI agents
• agent-desktop: Open-source agent computer use