NVIDIA Agent Toolkit Expands With Omniverse Libraries for Physical AI Simulation

NVIDIA Agent Toolkit Gets Physical AI Capabilities

NVIDIA announced at SIGGRAPH that its NVIDIA Agent Toolkit now includes Omniverse libraries, a collection of software components designed to bring physical AI capabilities into existing applications1

1

. The expansion marks a significant step in enabling AI agents to prepare 3D content for simulation, addressing a critical need as robots, factories, and autonomous systems require extensive testing in virtual environments before real-world deployment. The new libraries are openly available on GitHub, making them accessible to developers across the industry1

1

.

Jensen Huang, NVIDIA's founder and CEO, emphasized the strategic importance of this development: "The physical AI era will be built in simulation first. NVIDIA Agent Toolkit with Omniverse libraries brings AI agents into the 3D tools developers already use, helping build the simulation-ready worlds where robots, factories and autonomous systems are trained and tested long before they reach the real world"1

1

.

Three Core Libraries Transform 3D Workflows

The Omniverse libraries introduce three key capabilities that extend AI agents into physical AI workflows. The ovrtx library enables RTX sensor simulation, generating camera, lidar, radar, and other sensor outputs from 3D scenes so developers and AI agents can test how physical AI systems perceive virtual environments2

2

. The ovphysx library delivers GPU-accelerated physics simulation, bringing realistic behavior to 3D scenes through properties such as collisions, mass, friction, and motion1

1

. Meanwhile, CAD-to-SimReady skills help convert computer-aided design data to SimReady assets built on OpenUSD, giving 3D content the properties needed for asset validation and virtual testing2

2

.

These tools address a fundamental challenge: preparing 3D content for simulation requires more than realistic visuals. Assets need the right structure, materials, scale, labels, sensors, and physical properties to function effectively in simulation-first development workflows1

1

.

Industry Adoption Accelerates

Major software makers are already integrating these capabilities. SideFX is exploring how AI agents can integrate Omniverse libraries into its Houdini procedural 3D content creation workflows, giving technical artists a path to generate, test physics, and prepare content for simulation1

1

. Kim Davidson, president and CEO of SideFX, noted that "procedural 3D creation is essential to building the complex, controllable worlds needed for simulation, robotics and industrial AI"1

1

.

PTC's Onshape CAD and product data management platform is using OpenUSD and ovrtx to connect cloud-native design workflows with physical simulation, helping product design content stay connected with CAD, PDM, collaboration, and simulation workflows2

2

. Neil Barua, president and CEO of PTC, emphasized that "engineering teams are seeking more connected ways to design, collaborate and simulate throughout the development process"1

1

.

A new blueprint for integrating Omniverse libraries in Blender is now available on GitHub, demonstrating workflows that can run on systems ranging from RTX-powered systems with RTX Spark to GB300-powered systems with DGX Station2

2

. Several startups, including ForgeCAD, Lightwheel, Moonlake AI, and Palatial, are among the first to adopt and build with these libraries1

1

.