NVIDIA just released its latest NVIDIA Omniverse platform. Announced at SIGGRAPH 2023, this is a very significant release. It offers new foundation applications and services for developers and industrial enterprises to enhance their 3D pipelines with generative AI and the OpenUSD framework. (see: Architosh, “Pixar, Apple, Adobe, Autodesk and NVIDIA Form Alliance for OpenUSD,” 3 Aug 2023).
Omniverse and OpenUSD
Designed to meet the needs of large-scale film and visual effects production, OpenUSD is Universal Scene Description. This is a high-performance extensible software platform for collaboratively constructing animated 3D scenes. News of a recently formed alliance around OpenUSD brings tech giants Apple, Adobe, and Autodesk into the fold with Pixar and NVIDIA to advance this critical new format.
NVIDIA’s vice president of Omniverse and simulation technology, Rev Lebaredian. Image: NVIDIA.
NVIDIA’s vice president of Omniverse and simulation technology, Rev Lebaredian, explains the significance of OpenUSD:
“Industrial enterprises are racing to digitalize their workflows, increasing the demand for OpenUSD-enabled, connected, interoperable, 3D software ecosystems. The latest Omniverse update lets developers tap generative AI through OpenUSD to enhance their tools, and it allows enterprises to build larger, more complex world-scale simulations as digital testing grounds for their industrial applications.”
This latest NVIDIA Omniverse enhances interoperability between software tools and data types for the building of virtual worlds. It offers a new portfolio of technologies and cloud application programming interfaces (APIs), including ChatUSD and RunUSD, along with a new NVIDIA OpenUSD Developer Program. The latest NVIDIA Omniverse updates also include enhancements to Omniverse Kit. This is the engine used for developing native OpenUSD applications and extensions.
Other enhancements
- Boosted efficiency and user experience. — New rendering optimizations take full advantage of the NVIDIA Ada Lovelace architecture enhancements in NVIDIA RTX™ GPUs.
- Native RTX-powered spatial integration. — New extended-reality (XR) developer tools let users build spatial-computing options natively into their Omniverse-based applications, giving users the flexibility to experience their 3D projects and virtual worlds however they please.
- New developer templates and resources. — New application and experience templates provide developers getting started with OpenUSD and Omniverse a significant head-start with minimal coding.
- New modular app building. — A new Omniverse Kit Extension Registry, a central repository for accessing, sharing, and managing Omniverse extensions, lets developers easily turn functionality on-and-off in their application. This makes it easy to build custom apps from over 600 core Omniverse extensions provided by NVIDIA.
Platform updates showcased in Omniverse Foundation applications
- Omniverse USD Composer. — This enables 3D users to collect and curate lots of OpenUSD-based scenes.
- Omniverse Audio2Face. — This provides access to generative AI application programming interfaces. These create realistic facial animations and gestures from an audio file. Moreover, it now includes multilingual support and a new female base model.
OpenUSD expands Omniverse ecosystem
NVIDIA also announced a broad range of frameworks, resources, and services for developers and companies to accelerate their adoption of OpenUSD. It also announced its new Omniverse Cloud APIs. NVIDIA built these specifically for developers to implement and deploy OpenUSD pipelines and applications. For example, ChatUSD is a large language model copilot for developers that can answer OpenUSD knowledge questions or generate Python-USD code scripts.
Expanding their collaboration across Adobe Substance 3D, generative AI and OpenUSD initiatives, Adobe and NVIDIA announced plans to make Adobe Firefly available as APIs in Omniverse. This will enable developers and creators to enhance their design processes.
Wonder Dynamics is connected to Omniverse with new OpenUSD export support through its AI platform Wonder Studio. This automatically animates, lights, and composes computer-generated characters into live-action scenes. New OpenUSD export support also enables artists to generate and export a complete 3D scene, all from a single-camera video.
Import / Export
The main purpose of OpenUSD is to allow a wide variety of design software to share data easily. Consequently, Omniverse users can now build content, experiences, and applications that are compatible with other OpenUSD spatial computing platforms.
OpenUSD in action: housing development and car composite. Screengrab: NVIDIA.
For example, the latest NVIDIA Omniverse readily imports Luma AI’s reality-capture models in USDZ format. Tools from avatar company Convai and character-engine company Inworld AI are also connected to Omniverse.
ARKit and RealityKit.
The latest NVIDIA Omniverse also supports the Khronos Group’s OpenXR open standard. Consequently, allowing Omniverse to use more headsets from manufacturers such as HTC VIVE, Magic Leap, and Varjo.
With AI tools like Convai, creators can add characters to their digital twin environments. As a result, these characters can provide relevant information about the environment and objects, like a tour guide or a virtual robot.
Move AI enables single-camera motion capture with the Move One app, which can generate 3D character animations. It can then export these via OpenUSD and deploy them in Omniverse.
Users of SideFX Houdini can also now load Houdini Digital Assets directly into the Omniverse viewport. Moreover, the Cesium extension for Omniverse enables 3D Tiles, an open standard for streaming massive geospatial datasets in virtual worlds, including those supported by OpenUSD.
Cadence DataCenter Design Software is also now available via Omniverse. As a result, users can see computational fluid dynamics simulations in the complete context of their digital twin. The Blackshark.AI world digital twin platform also connects to Omniverse now.
NVIDIA’s customers using Omniverse for digitalization
- Boston Dynamics AI Institute is using Omniverse to simulate robots and their interactions. This is to enable the design of new robotics and control systems.
- Continental, is using Omniverse in its mobile robots business. As a result, it can generate physically accurate synthetic data at scale to train computer-vision AI models and perform system-integration testing.
- Marks Design, a brand design and experience agency, is using Omniverse. Together with OpenUSD, it streamlines collaboration and improves its animation, rendering, and visualization workflows.
- Volvo Cars has transitioned its digital twin to OpenUSD, using Omniverse to create immersive visualizations. Consequently, this helps customers make better-informed online purchasing decisions.
New Omniverse partners and systems
NVIDIA is collaborating with global systems manufacturers to bring RTX workstations optimally configured for Omniverse to millions of users. These new workstations feature up to four NVIDIA RTX 6000 Ada Generation GPUs. These are also bundled with NVIDIA Omniverse Enterprise software.
MORE: BOXX Upgrades AI Workstation Module with new NVIDIA L40S GPUs
As a result, this accelerates OpenUSD world-building, generative AI-enhanced collaborative design. Omniverse users can also take advantage of the new NVIDIA L40S GPU. This is a powerful, universal data center GPU that speeds up the most graphics-intensive workloads.
Availability
- This latest NVIDIA Omniverse release is now available in beta as a free download, from here.
- Latest NVIDIA Omniverse release will also be available shortly in Omniverse Enterprise, which you can learn more about here.
- To learn more about the latest NVIDIA Omniverse and watch NVIDIA CEO Jensen Huang’s SIGGRAPH keynote address, please click here.
- To learn more about SIGGRAPH 2023, please click here.