At NVIDIA’s GTC this week (which started yesterday), the company announced that a new framework built on Omniverse Cloud APIs can now stream digital twin and other 3D scene data to Apple’s new Vision Pro headset.
USD and Vision Pro
OpenUSD (open Universal Scene Description) has further democratized the USD format Pixar invented years ago, and the latest NVIDIA technologies with Omniverse Cloud APIs means software developers can now send their industrial scenes from the content creation applications of their choice to the NVIDIA Graphics Delivery Network (GDN), a global network of data centers that are “graphics-ready” and can stream high-fidelity 3D experiences to the Apple Vision Pro.
Apple Vision Pro can now be harnessed in the NVIDIA Omniverse platform. (Image: NVIDIA).
At the conference, a presentation showcased a demo of a designer wearing the Vision Pro while using a car configurator application developed by GCI studio Katana on the Omniverse platform. The designer works through the various options for the car inside the configurator application on the Vision Pro, selecting things like finishes and paint color. Then, the designer sits on a chair while the entire car is built around her—inside the Apple Vision Pro—as if sitting in the driver’s seat.
By leveraging the power of spatial computing platforms like the Apple Vision Pro and streaming high-fidelity advanced 3D scenes to the headset, spatial computing takes the user experience (UX) to unprecedented levels of realism and virtual experiences.
How It Works
The power of NVIDIA’s RTX cloud rendering networks delivers spatial computing experiences with just the device and an internet connection. The Vision Pro is an independent computer and does not rely on a secondary computer.
This cloud-based approach directly connected to the Vision Pro allows real-time physically based renderings to be seamlessly streamed to the Apple Vision Pro. High-fidelity visual imagery with uncompromising detail can be streamed, including massive engineering datasets.
Apple Vision Pro is the first untethered device that allows for enterprise customers to realize their work without compromise. We look forward to our customers having access to these amazing tools.
“The breakthrough ultra-high-resolution displays of Apple Vision Pro, combined with photorealistic rendering of OpenUSD content streamed from NVIDIA accelerated computing, unlocks an incredible opportunity for the advancement of immersive experiences,” said Mike Rockwell, vice president of the Vision Products Group at Apple. “Spatial computing will redefine how designers and developers build captivating digital content, driving a new era of creativity and engagement.”
The Vision Pro allows the user to virtually sit inside the car they have just configured using an Omniverse-powered configurator app. (Image: NVIDIA)
“Apple Vision Pro is the first untethered device that allows for enterprise customers to realize their work without compromise,” said Rev Lebaredian, vice president of simulation at NVIDIA. “We look forward to our customers having access to these amazing tools.”
The workflow shown and discussed above leverages a new hybrid rendering—combining remote (cloud) and local (Vision Pro) rendering on the device. Users can render fully interactive scenes in a single application from Apple’s native SwiftUI and Apple Reality Kit with the Omniverse RTX Renderer streaming from the GDN.