At NVIDIA GTC this year a significant theme was Physical AI the kind of intelligence that enables robots to operate effectively in the actual physical world. Nvidia showcased a suite of technologies designed to help robots acquire the skills needed for this tangible interaction.
A key component of Nvidia’s approach is Isaac Groot N1 an open source foundation model. This model provides robots with a pre trained base of fundamental abilities. It is structured with two systems a fast system for immediate action execution and a slow system dedicated to planning more complex sequences of actions.
The next critical step is teaching these robots new specific tasks. This presents a challenge as it requires vast and diverse datasets especially if the robot needs to perform the learned task reliably under various real world conditions.
Nvidia is addressing this data challenge through its Omniverse platform and Cosmos technology. As explained by Akquil Doka Senior Product Manager for Nvidia Omniverse Omniverse serves as a platform that integrates data from different sources to build physically accurate digital twins simulations of real environments.
The data generated within Omniverse then flows through Cosmos. Cosmos plays a vital role not only in augmenting data for photo realism reducing the simulation to real world gap but also in enabling the creation of exponentially large datasets. This means transforming maybe a few thousand initial images into millions thereby providing the crucial data diversity required to train models that can generalize and perform well across different scenarios. One method for gathering initial training data involves teleoperation where a human guides the robot demonstrating the task which is then scaled up using this synthetic data.
For synthetic data to be truly applicable to the real world it must accurately reflect the laws of physics. Recognizing this need Nvidia announced a significant development at GTC Newton a new physics engine. Newton is the result of a notable partnership between DeepMind Disney Research and Nvidia. Importantly like Isaac Groot N1 Newton is being made open source. This move aims to not only provide developers with this powerful tool but also to foster contributions back to the engine from the wider community.
The broader context driving much of this work on humanoid robots is the widely discussed concern about looming labor shortages. Jensen Huang highlighted the clear reality that the world faces a severe shortage of human laborers projecting a deficit potentially reaching 50 million workers by the end of this decade. He even posed the idea that eventually we might effectively be “paying” robots to work, though a representative from Nvidia later clarified this cost model.
Nvidia explained the cost is better understood not as a salary but more like a subscription service for an autonomous vehicle. You purchase the car and then pay for the specific services or capabilities you want it to perform. They believe a similar model will apply to robots you buy the robot platform and then subscribe to the specific skills or services it needs to carry out.
The technologies showcased at GTC particularly Isaac Groot N1 Omniverse Cosmos and the new Newton physics engine demonstrate Nvidia’s focus on providing the fundamental building blocks for training physically capable robots. This work is aimed at enabling robots especially humanoids to learn and perform complex tasks interacting effectively with our physical world and potentially helping address future labor needs.