NVIDIA Open-Sources Four Cosmos 3 Nano World-Simulation Models With Joint Camera-and-LiDAR Generation for Robots and AVs

NVIDIA has released four new open-weight checkpoints in its Cosmos 3 Nano family for interactive world simulation, published on Hugging Face on November 5 alongside a companion developer blog post. The four models — Cosmos3-Nano-Sim-Auto, Cosmos3-Nano-Sim-Transfer, Cosmos3-Nano-Sim-Bimanual and Cosmos3-Nano-Transfer-Auto — are each listed at around 15 billion parameters and ship under the OpenMDW 1.1 license, which permits both commercial and non-commercial use, with post-training recipes and a technical report included.

The division of labor among the four is worth understanding, because they target different points on the simulation spectrum. Cosmos3-Nano-Transfer-Auto is the autonomous-driving heavyweight: it generates synchronized video across eleven cameras, including fisheye lenses, with joint RGB and LiDAR generation and HD-map controls — a full multi-sensor reconstruction of a driving scene from a single pass. Cosmos3-Nano-Sim-Auto is its causally distilled streaming sibling, built to run in a closed loop with few denoising steps so an agent can act, observe and act again.

The other two serve robotics. Cosmos3-Nano-Sim-Transfer converts control signals from a physics simulator — depth maps, segmentation, scene edges — into realistic color video guided by a text prompt, letting researchers keep the simulator responsible for motion while the model supplies photorealistic observation. Cosmos3-Nano-Sim-Bimanual instead predicts the next visual observation directly from an image plus the actions of a two-arm robot, supporting the YAM and AgiBot embodiments with action formats that specify coordinate frames, rotations and gripper calibration.

Architecturally, all four are built on a Mixture-of-Transformers design that connects separate text-processing and image-generation networks through attention, with temporal causal attention so each generated clip reads the history that came before it. The training recipe runs in four stages: adaptation to complete recorded clips, teacher forcing on earlier observations, consistency distillation down to four denoising steps, and a final self-forcing stage that trains the model on its own generated history to prevent early errors from compounding. NVIDIA says its published vLLM-Omni setup was tested on a single GB200 GPU, though the company notes that is not a minimum-memory specification.

What the release does not include is a benchmark comparison. NVIDIA provides no leaderboard numbers against competing world models such as Google's Genie, GAIA or World Labs' offerings, and acknowledges on its platform page that measuring how well these models support policy learning and evaluation — and finding where they fall short — remains work to be done. Buyers should treat the capability descriptions as the company's own framing rather than independently verified results.

The release extends an aggressive open-weights cadence for NVIDIA's physical-AI push. In late October the company open-sourced Cosmos3-H-Dreams, a surgical-domain world model trained on real operating-room footage that runs live on a single GPU. The new Nano batch pushes the same strategy toward the two biggest physical-AI markets: autonomous driving, where synthetic multi-sensor data is scarce and expensive, and robot learning, where closed-loop simulation is the only affordable way to give a policy millions of practice attempts.

For researchers and driving startups, the practical pitch is a customization path: fine-tune Transfer-Auto on local road, weather or signage data, then distill it into Sim-Auto for interactive closed-loop training. With commercial use explicitly permitted and the recipes public, the real cost of adoption shifts from licensing to hardware. Whether the open models match the closed frontier on visual fidelity is unmeasured, but the barrier to building a private world simulator just dropped again.