NVIDIA Unveils Comprehensive AI Stack for Next-Generation Humanoid Robots

NVIDIA unveils brain-and-body stack to train next-gen humanoids

Together, these technologies give researchers and developers a unified, open, and accelerated robotics stack. The tools promise to speed up iteration, standardize testing, unify training with on-robot inference, and help robots transfer skills more safely from simulation to the real world. "Humanoids are the next frontier of physical AI, requiring the ability to reason, adapt and act safely in an unpredictable world," said Rev Lebaredian, vice president of Omniverse and simulation technology at NVIDIA. "With these latest updates, developers now have the three computers to bring robots from research into everyday life -- with Isaac GR00T serving as robot's brains, Newton simulating their body and NVIDIA Omniverse as their training ground." The updates also underscore NVIDIA's ambition to define the standard platform for building general-purpose robots, especially humanoids, which demand precise physics and complex reasoning to function in human environments.

With new open models and simulation libraries, Nvidia aims to accelerate robotics R&D - SiliconANGLE

With new open models and simulation libraries, Nvidia aims to accelerate robotics R&D As Nvidia Corp. announced new robotics innovations at last week's Conference on Robot Learning in South Korea, the company continues to extend its product line with new capabilities and enhancements. Nvidia announcements at CoRL included: "Humanoids are the next frontier of physical AI, requiring the ability to reason, adapt and act safely in an unpredictable world," Rev Lebaredian, vice president of Omniverse and simulation technology at Nvidia, said in an analyst briefing before the conference. "With these latest updates, developers now have the three computers to bring robots from research into everyday life -- with Isaac GR00T serving as the robot's brains, Newton simulating their body, and Nvidia Omniverse as their training ground." Lebaredian said nearly half of the papers accepted at CoRL cited the use of Nvidia technology, including accelerated computing platforms and software libraries. Nvidia has taken an architectural approach to physical AI with three computer system to build robots. Blackwell DGX or HGX AI systems to train the physical AI model, RTX PRO servers and Omniverse with Cosmos to simulate and test physical AI models, and Jetson AGX Thor powered by Blackwell for on-robot inference. "We build the three computers, plus the open models, open-source simulation frameworks and data pipelines that run on these computers for physical AI developers," explained Lebaredian. The updates to GR00T N1.6 include using Cosmos Reason as its long-thinking brain. Cosmos Reason is an open, customizable reasoning VLM, or vision language model, for physical AI. It will bring human-like reasoning to humanoids, allowing them to break down complex instructions and execute tasks using prior knowledge and common sense. It will also let humanoids move and handle objects simultaneously with more torso and arm freedom to complete tough tasks like opening heavy doors while carrying items. "Reasoning is the best tool we have currently for extending from the set of things that we fed into the AIs for their training into novel new paradigms and environments," Lebaredian said. Unlike large language models that can be trained using human knowledge from the internet, such data doesn't exist for training physical AI models according to Lebaredian. "Real-world data is costly and potentially dangerous to capture, and pre-training only goes so far," he explained. "To train models like GR00T, we need a scalable and cost-effective way to generate large, diverse and physically accurate data." To provide that training, Nvidia announced that new versions of the Cosmos-Predict and Cosmos-Transfer World Foundation models will be available soon. Cosmos-Predict is a breakthrough in robot training as it generates future states from an initial state, providing the data that's lacking today. This new release unifies three separate models into one, cutting post-training time, which reduces complexity and that lowers compute cost. It delivers higher quality than previous versions, as well as open-source models of similar size. It now supports multiview outputs for multisensor robots and autonomous vehicles, and can produce videos of up to 30 seconds. "Cosmos-Transfer performs world-to-world style transfer, and the latest version is 3.5 times smaller than before," Lebaredian continued. "This smaller footprint lowers compute cost and makes it easier for developers to augment and scale training data. Together, these models enable the generation of hundreds of virtual sensor-rich environments for robot training, reducing reliance on real-world data collection." As "the era of robot reasoning" begins, Lebaredian said "robots need a supercomputer that can power the entire system from brain to body." Nvidia's announcement last month of Jetson Thor, which is designed for physical AI and robotics and is powered by a Nvidia Blackwell GPU and 128 gigabytes of memory, delivers "the AI performance to run the latest models, including the Isaac GR00T and Cosmos World Foundation models," he said. In his briefing, Lebaredian addressed how Nvidia's open-source approach to Newton and Nvidia Isaac GR00T benefits the robotics community, particularly researchers and startups. "The very nature of how research happens, how we advance the frontier of human knowledge, is about openness," he said. "It's about sharing information between all of the researchers so they can advance together. The only way to really do that well in the computing world is by also sharing the same software, algorithms and techniques that are both developed and the tools and pipelines behind them used to actually conduct the research." With robotics and physical AI, Larabedian added, "we're still on the frontiers. It's critical that we advance these frontiers together. To do that, we have to contribute to open source and do this all in the open so everyone can move together. Nvidia, being in the position that it's in, can disproportionately help by putting our vast resources behind the software development and the technology necessary to power all of this, and we're doing so."

New NVIDIA Models Helps Robots Learn, Reason, and Act in the Real World | AIM

NVIDIA also unveiled the GB200 NVL72 system, RTX PRO servers and Jetson Thor for real-time on-robot inference. NVIDIA has released new open models and simulation libraries to support global robotics research and development. The announcement, made at the Conference on Robot Learning (CoRL) in Seoul, includes the open-source Newton Physics Engine, the Isaac GR00T N1.6 foundation model and new Cosmos world foundation models. The updates aim to accelerate the way robots learn, reason, and transfer skills from virtual to real environments. Rev Lebaredian, vice president of Omniverse and simulation technology at NVIDIA, said, "Humanoids are the next frontier of physical AI, requiring the ability to reason, adapt and act safely in an unpredictable world." The Newton Physics Engine, developed with Google DeepMind and Disney Research, is now available in NVIDIA Isaac Lab. Managed by the Linux Foundation, the engine supports complex actions such as walking on uneven ground and handling delicate objects. ETH Zurich, Technical University of Munich and Peking University are among the first adopters. NVIDIA has also announced Isaac Lab - Arena, an open-source policy evaluation framework codeveloped with Lightwheel. The tool will enable large-scale testing of robotic skills in diverse simulated environments. The Isaac GR00T N1.6 model integrates NVIDIA's Cosmos Reason, which helps robots turn vague instructions into step-by-step plans using prior knowledge and physics-based reasoning. The model, available on Hugging Face, also supports multi-task operations such as opening heavy doors. NVIDIA's Cosmos world foundation models, which have been downloaded more than 3 million times, have been updated to generate large-scale training data. Cosmos Predict 2.5 and Cosmos Transfer 2.5, due soon, will offer longer video generation, multi-view outputs and faster synthetic data creation. Boston Dynamics used NVIDIA's new grasping workflow in Isaac Lab 2.3 to train its Atlas robots to improve manipulation skills. Other companies adopting NVIDIA's Isaac and Omniverse platforms include Agility Robotics, Figure AI, Franka Robotics, Techman Robot and Solomon. NVIDIA also unveiled new AI infrastructure, including the GB200 NVL72 system, RTX PRO servers and Jetson Thor for real-time on-robot inference. These tools are being adopted by partners such as Figure AI, Meta, Google DeepMind and the RAI Institute. Lebaredian added, "With these latest updates, developers now have the three computers to bring robots from research into everyday life, with Isaac GR00T serving as the robot's brain, Newton simulating their body and NVIDIA Omniverse as their training ground."