XR-based digital twin of Tampa built by IMERZA using Unreal Engine

Continued from page 1

IMERZA’s Process—Unreal Engine

While it took one year to complete the entire project, IMERZA wrote six custom applications including interactive touchscreen kiosks, a video wall application, an Apple iPad application, a custom data aggregator, a content management system (CMS), and a projection-mapped content application.

The Unreal Engine is at the core of four of the six applications. All of the applications communicate with each other via the IMERZA API. For example, the SPP team can select an individual building and show that building’s cone-of-view on the physical model, while simultaneously showing the views from every view of that building on the surrounding wall screens. They can also rotate that view around 360 degrees by turning a compass on the custom iPad app. Both the projection model and the video walls rotate together in perfect sync.

Advertising

The SPP team can upload a potential advertiser’s collateral to their CMS and have the system automatically apply it to all the billboards and digital signage locations throughout the Tampa environment, both on the scale model and via the video wall imagery as well.

Advertising can be displayed on the model and witnessed in context through the rendered real-time views.

This means you can have the orientation nature of the physical model and see where all advertising will go as well as the “experience” of encountering that advertising in context. This is fundamentally about simulation.

“Features like this could only be possible with real-time rendering,” says Dorian Vee, Co-Founder, and CTO of IMERZA. “Unreal Engine renders all of the content, and technologies such as nDisplay and NVIDIA Quadro RTX cards enable us to keep the multiple PCs frame-locked and rendering 12 separate cameras to the 12 projectors in real-time.”

Unreal Engine

The team opted for Unreal Engine to power its applications because it needed a solution that could produce stunning visuals while also providing the ability to connect to multiple third-party tools and handle complex datasets. “The choice to use Unreal Engine was really a no-brainer,” recalls Vee. “Unreal Engine allows our artists to create beautiful visuals faster utilizing real-time ray tracing, and the open-source code and access to technologies such as nDisplay allowed us to customize the engine to suit our needs and innovate quickly.”

Time-of-Day rendering is fully supported thanks to the Unreal Engine the rendering quality is very high.

One of the most impressive aspects of the project from a visual standpoint is the clarity of the projections the team was able to achieve. This was no walk in the park. “The biggest challenge was rendering 12 different cameras to 12 projectors and ensuring everything, down to each particle, was absolutely frame-locked,” explains Vee. “If it wasn’t frame-locked, the projection would be a blurry mess.”

Unreal Engine, NVIDIA Quadro cards, and nDisplay technology were the key to frame-locking the 24 million pixels being projected and rendered at over 90 frames per second.

Real-time views from any building in the Tampa city area can be projected onto the walls around the room while highlighting the building in the 3D model and the view cone.

For the video walls and touchscreens, IMERZA also used Quadro RTX 6000 cards, leveraging real-time ray tracing to give extra realism to the scenes. In order to keep everything running efficiently at 4K, the team used Blueprint-driven logic to toggle between RTX-GI and screen-space GI based on view distance. RTX capability can be turned on and off within materials using dynamic RTX reflections based on a material node, driven by camera visibility and distance.

Real-time Technology

Interactive workflows like those behind the Water Street Tampa Marketing Center project are ironing out some of the friction that exists in the traditional architectural design process. “I could go on for days about this, but I’ll keep it simple,” says Vee. “The majority of people have an extremely difficult time visualizing design. Real-time technology allows everyone to see the same thing and make decisions faster. This translates to reduced costs, fewer change-orders, and overall better design.”

Vee is equally emphatic about the overall impact real-time technology is having on real estate and urban development. “It isn’t often something comes along that can revolutionize an industry. Something that revolutionizes several industries at once happens maybe once a century,” he says. “Unreal Engine is doing just that. AEC, film production, product design, vehicle design, training, and manufacturing—even creating new industries. It’s exciting, and I feel lucky to be a part of it.”

next page: Further Resources