Curated by THEOUTPOST
On Thu, 24 Oct, 4:05 PM UTC
2 Sources
[1]
The Three Computer Solution: Powering the Next Wave of AI Robotics
ChatGPT marked the big bang moment of generative AI. Answers can be generated in response to nearly any query, helping transform digital work such as content creation, customer service, software development and business operations for knowledge workers. Physical AI, the embodiment of artificial intelligence in humanoids, factories and other devices within industrial systems, has yet to experience its breakthrough moment. This has held back industries such as transportation and mobility, manufacturing, logistics and robotics. But that's about to change thanks to three computers bringing together advanced training, simulation and inference. For 60 years, "Software 1.0" - serial code written by human programmers - ran on general-purpose computers powered by CPUs. Then, in 2012, Alex Krizhevsky, mentored by Ilya Sutskever and Geoffrey Hinton, won the ImageNet computer image recognition competition with AlexNet, a revolutionary deep learning model for image classification. This marked the industry's first contact with AI. The breakthrough of machine learning - neural networks running on GPUs - jump-started the era of Software 2.0. Today, software writes software. The world's computing workloads are shifting from general-purpose computing on CPUs to accelerated computing on GPUs, leaving Moore's law far behind. With generative AI, multimodal transformer and diffusion models have been trained to generate responses. Large language models are one-dimensional, able to predict the next token, in modes like letters or words. Image- and video-generation models are two-dimensional, able to predict the next pixel. None of these models can understand or interpret the three-dimensional world. And that's where physical AI comes in. Physical AI models can perceive, understand, interact with and navigate the physical world with generative AI. With accelerated computing, multimodal physical AI breakthroughs and large-scale physically based simulations are allowing the world to realize the value of physical AI through robots. A robot is a system that can perceive, reason, plan, act and learn. Robots are often thought of as autonomous mobile robots (AMRs), manipulator arms or humanoids. But there are many more types of robotic embodiments. In the near future, everything that moves, or that monitors things that move, will be autonomous robotic systems. These systems will be capable of sensing and responding to their environments. Everything from surgical rooms to data centers, warehouses to factories, even traffic control systems or entire smart cities will transform from static, manually operated systems to autonomous, interactive systems embodied by physical AI. Humanoid robots are an ideal general-purpose robotic manifestation because they can operate efficiently in environments built for humans, while requiring minimal adjustments for deployment and operation. The global market for humanoid robots is expected to reach $38 billion by 2035, a more than sixfold increase from the roughly $6 billion for the period forecast nearly two years ago, according to Goldman Sachs. Researchers and developers around the world are racing to build this next wave of robots. To develop humanoid robots, three accelerated computer systems are required to handle physical AI and robot training, simulation and runtime. Two computing advancements are accelerating humanoid robot development: multimodal foundation models and scalable, physically based simulations of robots and their worlds. Breakthroughs in generative AI are bringing 3D perception, control, skill planning and intelligence to robots. Robot simulation at scale lets developers refine, test and optimize robot skills in a virtual world that mimics the laws of physics - helping reduce real-world data acquisition costs and ensuring they can perform in safe, controlled settings. NVIDIA has built three computers and accelerated development platforms to enable developers to create physical AI. First, models are trained on a supercomputer. Developers can use NVIDIA NeMo on the NVIDIA DGX platform to train and fine-tune powerful foundation and generative AI models. They can also tap into NVIDIA Project GR00T, an initiative to develop general-purpose foundation models for humanoid robots to enable them to understand natural language and emulate movements by observing human actions. Second, NVIDIA Omniverse, running on NVIDIA OVX servers, provides the development platform and simulation environment for testing and optimizing physical AI with application programming interfaces and frameworks like NVIDIA Isaac Sim. Developers can use Isaac Sim to simulate and validate robot models, or generate massive amounts of physically-based synthetic data to bootstrap robot model training. Researchers and developers can also use NVIDIA Isaac Lab, an open-source robot learning framework that powers robot reinforcement learning and imitation learning, to help accelerate robot policy training and refinement. Lastly, trained AI models are deployed to a runtime computer. NVIDIA Jetson Thor robotics computers are specifically designed for compact, on-board computing needs. An ensemble of models consisting of control policy, vision and language models composes the robot brain and is deployed on a power-efficient, on-board edge computing system. Depending on their workflows and challenge areas, robot makers and foundation model developers can use as many of the accelerated computing platforms and systems as needed. Robotic facilities result from a culmination of all of these technologies. Manufacturers like Foxconn or logistics companies like Amazon Robotics can orchestrate teams of autonomous robots to work alongside human workers and monitor factory operations through hundreds or thousands of sensors. These autonomous warehouses, plants and factories will have digital twins. The digital twins are used for layout planning and optimization, operations simulation and, most importantly, robot fleet software-in-the-loop testing. Built on Omniverse, "Mega" is a blueprint for factory digital twins that enables industrial enterprises to test and optimize their robot fleets in simulation before deploying them to physical factories. This helps ensure seamless integration, optimal performance and minimal disruption. Mega lets developers populate their factory digital twins with virtual robots and their AI models, or the brains of the robots. Robots in the digital twin execute tasks by perceiving their environment, reasoning, planning their next motion and, finally, completing planned actions. These actions are simulated in the digital environment by the world simulator in Omniverse, and the results are perceived by the robot brains through Omniverse sensor simulation. With sensor simulations, the robot brains decide the next action, and the loop continues, all while Mega meticulously tracks the state and position of every element within the factory digital twin. This advanced software-in-the-loop testing methodology enables industrial enterprises to simulate and validate changes within the safe confines of the Omniverse digital twin, helping them anticipate and mitigate potential issues to reduce risk and costs during real-world deployment. NVIDIA accelerates the work of the global ecosystem of robotics developers and robot foundation model builders with three computers. Universal Robots, a Teradyne Robotics company, used NVIDIA Isaac Manipulator, Isaac accelerated libraries and AI models, and NVIDIA Jetson Orin to build UR AI Accelerator, a ready-to-use hardware and software toolkit that enables cobot developers to build applications, accelerate development and reduce the time to market of AI products. RGo Robotics used NVIDIA Isaac Perceptor to help its wheel.me AMRs work everywhere, all the time, and make intelligent decisions by giving them human-like perception and visual-spatial information. Humanoid robot makers including 1X Technologies, Agility Robotics, Apptronik, Boston Dynamics, Fourier, Galbot, Mentee, Sanctuary AI, Unitree Robotics and XPENG Robotics are adopting NVIDIA's robotics development platform. Boston Dynamics is using Isaac Sim and Isaac Lab to build quadrupeds and humanoid robots to augment human productivity, tackle labor shortages and prioritize safety in warehouses. Fourier is tapping into Isaac Sim to train humanoid robots to operate in fields that demand high levels of interaction and adaptability, such as scientific research, healthcare and manufacturing. Using Isaac Lab and Isaac Sim, Galbot advanced the development of a large-scale robotic dexterous grasp dataset called DexGraspNet that can be applied to different dexterous robotic hands, as well as a simulation environment for evaluating dexterous grasping models. Field AI developed risk-bounded multitask and multipurpose foundation models for robots to safely operate in outdoor field environments, using the Isaac platform and Isaac Lab. The era of physical AI is here - and it's transforming the world's heavy industries and robotics.
[2]
The Three Computer Solution: Powering the Next Wave of AI Robotics
Your browser doesn't support HTML5 video. Here is a link to the video instead. ChatGPT marked the big bang moment of generative AI. Answers can be generated in response to nearly any query, helping transform digital work such as content creation, customer service, software development and business operations for knowledge workers. Physical AI, the embodiment of artificial intelligence in humanoids, factories and other devices within industrial systems, has yet to experience its breakthrough moment. This has held back industries such as transportation and mobility, manufacturing, logistics and robotics. But that's about to change thanks to three computers bringing together advanced training, simulation and inference. The Rise of Multimodal, Physical AI For 60 years, "Software 1.0" -- serial code written by human programmers -- ran on general-purpose computers powered by CPUs. Then, in 2012, Alex Krizhevsky, mentored by Ilya Sutskever and Geoffrey Hinton, won the ImageNet computer image recognition competition with AlexNet, a revolutionary deep learning model for image classification. This marked the industry's first contact with AI. The breakthrough of machine learning -- neural networks running on GPUs -- jump-started the era of Software 2.0. Today, software writes software. The world's computing workloads are shifting from general-purpose computing on CPUs to accelerated computing on GPUs, leaving Moore's law far behind. With generative AI, multimodal transformer and diffusion models have been trained to generate responses. Large language models are one-dimensional, able to predict the next token, in modes like letters or words. Image- and video-generation models are two-dimensional, able to predict the next pixel. None of these models can understand or interpret the three-dimensional world. And that's where physical AI comes in. Physical AI models can perceive, understand, interact with and navigate the physical world with generative AI. With accelerated computing, multimodal physical AI breakthroughs and large-scale physically based simulations are allowing the world to realize the value of physical AI through robots. A robot is a system that can perceive, reason, plan, act and learn. Robots are often thought of as autonomous mobile robots (AMRs), manipulator arms or humanoids. But there are many more types of robotic embodiments. In the near future, everything that moves, or that monitors things that move, will be autonomous robotic systems. These systems will be capable of sensing and responding to their environments. Everything from surgical rooms to data centers, warehouses to factories, even traffic control systems or entire smart cities will transform from static, manually operated systems to autonomous, interactive systems embodied by physical AI. The Next Frontier: Humanoids Robots Humanoid robots are an ideal general-purpose robotic manifestation because they can operate efficiently in environments built for humans, while requiring minimal adjustments for deployment and operation. The global market for humanoid robots is expected to reach $38 billion by 2035, a more than sixfold increase from the roughly $6 billion for the period forecast nearly two years ago, according to Goldman Sachs. Researchers and developers around the world are racing to build this next wave of robots. Three Computers to Develop Physical AI To develop humanoid robots, three accelerated computer systems are required to handle physical AI and robot training, simulation and runtime. Two computing advancements are accelerating humanoid robot development: multimodal foundation models and scalable, physically based simulations of robots and their worlds. Breakthroughs in generative AI are bringing 3D perception, control, skill planning and intelligence to robots. Robot simulation at scale lets developers refine, test and optimize robot skills in a virtual world that mimics the laws of physics -- helping reduce real-world data acquisition costs and ensuring they can perform in safe, controlled settings. NVIDIA has built three computers and accelerated development platforms to enable developers to create physical AI. First, models are trained on a supercomputer. Developers can use NVIDIA NeMo on the NVIDIA DGX platform to train and fine-tune powerful foundation and generative AI models. They can also tap into NVIDIA Project GR00T, an initiative to develop general-purpose foundation models for humanoid robots to enable them to understand natural language and emulate movements by observing human actions. Second, NVIDIA Omniverse, running on NVIDIA OVX servers, provides the development platform and simulation environment for testing and optimizing physical AI with application programming interfaces and frameworks like NVIDIA Isaac Sim. Developers can use Isaac Sim to simulate and validate robot models, or generate massive amounts of physically-based synthetic data to bootstrap robot model training. Researchers and developers can also use NVIDIA Isaac Lab, an open-source robot learning framework that powers robot reinforcement learning and imitation learning, to help accelerate robot policy training and refinement. Lastly, trained AI models are deployed to a runtime computer. NVIDIA Jetson Thor robotics computers are specifically designed for compact, on-board computing needs. An ensemble of models consisting of control policy, vision and language models composes the robot brain and is deployed on a power-efficient, on-board edge computing system. Depending on their workflows and challenge areas, robot makers and foundation model developers can use as many of the accelerated computing platforms and systems as needed. Building the Next Wave of Autonomous Facilities Robotic facilities result from a culmination of all of these technologies. Manufacturers like Foxconn or logistics companies like Amazon Robotics can orchestrate teams of autonomous robots to work alongside human workers and monitor factory operations through hundreds or thousands of sensors. These autonomous warehouses, plants and factories will have digital twins. The digital twins are used for layout planning and optimization, operations simulation and, most importantly, robot fleet software-in-the-loop testing. Built on Omniverse, "Mega" is a blueprint for factory digital twins that enables industrial enterprises to test and optimize their robot fleets in simulation before deploying them to physical factories. This helps ensure seamless integration, optimal performance and minimal disruption. Mega lets developers populate their factory digital twins with virtual robots and their AI models, or the brains of the robots. Robots in the digital twin execute tasks by perceiving their environment, reasoning, planning their next motion and, finally, completing planned actions. These actions are simulated in the digital environment by the world simulator in Omniverse, and the results are perceived by the robot brains through Omniverse sensor simulation. With sensor simulations, the robot brains decide the next action, and the loop continues, all while Mega meticulously tracks the state and position of every element within the factory digital twin. This advanced software-in-the-loop testing methodology enables industrial enterprises to simulate and validate changes within the safe confines of the Omniverse digital twin, helping them anticipate and mitigate potential issues to reduce risk and costs during real-world deployment. Empowering the Developer Ecosystem With NVIDIA Technology NVIDIA accelerates the work of the global ecosystem of robotics developers and robot foundation model builders with three computers. Universal Robots, a Teradyne Robotics company, used NVIDIA Isaac Manipulator, Isaac accelerated libraries and AI models, and NVIDIA Jetson Orin to build UR AI Accelerator, a ready-to-use hardware and software toolkit that enables cobot developers to build applications, accelerate development and reduce the time to market of AI products. RGo Robotics used NVIDIA Isaac Perceptor to help its wheel.me AMRs work everywhere, all the time, and make intelligent decisions by giving them human-like perception and visual-spatial information. Humanoid robot makers including 1X Technologies, Agility Robotics, Apptronik, Boston Dynamics, Fourier, Galbot, Mentee, Sanctuary AI, Unitree Robotics and XPENG Robotics are adopting NVIDIA's robotics development platform. Boston Dynamics is using Isaac Sim and Isaac Lab to build quadrupeds and humanoid robots to augment human productivity, tackle labor shortages and prioritize safety in warehouses. Fourier is tapping into Isaac Sim to train humanoid robots to operate in fields that demand high levels of interaction and adaptability, such as scientific research, healthcare and manufacturing. Using Isaac Lab and Isaac Sim, Galbot advanced the development of a large-scale robotic dexterous grasp dataset called DexGraspNet that can be applied to different dexterous robotic hands, as well as a simulation environment for evaluating dexterous grasping models. Field AI developed risk-bounded multitask and multipurpose foundation models for robots to safely operate in outdoor field environments, using the Isaac platform and Isaac Lab. The era of physical AI is here -- and it's transforming the world's heavy industries and robotics.
Share
Share
Copy Link
NVIDIA introduces a three-computer solution to advance physical AI and robotics, combining training, simulation, and runtime systems to revolutionize industries from manufacturing to smart cities.
While generative AI has transformed digital work, physical AI - the embodiment of artificial intelligence in robots and industrial systems - is on the cusp of a breakthrough. NVIDIA is leading this charge with a three-computer solution that promises to revolutionize industries such as transportation, manufacturing, and logistics 12.
The journey of AI has evolved from traditional programming (Software 1.0) to machine learning (Software 2.0), and now to physical AI. This progression has seen a shift from CPU-based general-purpose computing to GPU-accelerated computing, surpassing Moore's law 12.
Physical AI models are designed to perceive, understand, and interact with the three-dimensional world, unlike their one-dimensional (language) or two-dimensional (image) counterparts. This advancement is set to transform static, manually operated systems into autonomous, interactive systems across various sectors 12.
NVIDIA's approach to developing physical AI and robotics involves three key components:
Training Supercomputer: Utilizing the NVIDIA DGX platform and NeMo framework, developers can train and fine-tune powerful foundation and generative AI models. The company's Project GR00T aims to develop general-purpose foundation models for humanoid robots 12.
Simulation Environment: NVIDIA Omniverse, running on OVX servers, provides a development platform and simulation environment. Tools like Isaac Sim allow developers to test and optimize robot models in physically accurate virtual worlds 12.
Runtime Computer: Trained AI models are deployed on NVIDIA Jetson Thor robotics computers, designed for compact, on-board computing needs. These systems run an ensemble of control policy, vision, and language models that form the robot's brain 12.
Humanoid robots are emerging as an ideal general-purpose robotic manifestation, capable of operating efficiently in human-built environments. Goldman Sachs predicts the global market for humanoid robots to reach $38 billion by 2035, a significant increase from earlier forecasts 12.
This technological advancement is expected to transform various sectors:
While the potential of physical AI is immense, challenges remain in areas such as 3D perception, control, and skill planning. However, breakthroughs in generative AI and large-scale physically based simulations are accelerating development, reducing real-world data acquisition costs, and ensuring safe testing environments 12.
As the world moves towards autonomous robotic systems, NVIDIA's three-computer solution stands at the forefront, promising to unlock the full potential of physical AI and usher in a new era of robotics across industries.
Reference
[1]
[2]
Nvidia introduces Cosmos, a suite of world foundation models designed to bring generative AI capabilities to robotics and autonomous vehicles, potentially revolutionizing the development of physical AI systems.
15 Sources
15 Sources
Nvidia is pioneering spatial AI and the Omniverse platform, aiming to bring AI into the physical world through digital twins, robotics, and intelligent spaces. This technology could revolutionize industries from manufacturing to urban planning.
2 Sources
2 Sources
Nvidia introduces Isaac GR00T Blueprint at CES 2025, revolutionizing humanoid robotics development through synthetic data generation and imitation learning, leveraging Apple Vision Pro for motion capture.
3 Sources
3 Sources
NVIDIA announces new generative AI models and blueprints for Omniverse, expanding its integration into physical AI applications like robotics and autonomous vehicles. The company also introduces early access to Omniverse Cloud Sensor RTX for smarter autonomous machines.
4 Sources
4 Sources
NVIDIA's CEO Jensen Huang predicts widespread AI adoption and introduces 'Physical AI' at SIGGRAPH 2023, signaling a new era of AI-powered technology across various sectors.
2 Sources
2 Sources
The Outpost is a comprehensive collection of curated artificial intelligence software tools that cater to the needs of small business owners, bloggers, artists, musicians, entrepreneurs, marketers, writers, and researchers.
© 2025 TheOutpost.AI All rights reserved