Nvidia’s Project Holodeck: Photoreal Graphics in Shared VR

At Nvidia’s GPU Technology Conference, the company’s chief executive Jen-Hsun Haung introduced Project Holodeck, which aims to provide an experimental multi-user virtual environment with real-time photorealistic graphics and real-world physics. The new technology, which uses Epic’s Unreal Engine 4 and Nvidia’s GameWorks, VRWorks and DesignWorks, is targeted at design engineers and their collaborators. Nvidia’s Project Holodeck demo involved Koenigsegg Automotive, a Swedish company that makes exotic sports cars.

According to Tom’s Hardware, Christian Koenigsegg “joined the presentation via VR to show off his company’s latest creation, the Koenigsegg Regera hypercar.” In the demo, four robot avatars floated over a 3D grid space, and then a wireframe silhouette of the hypercar appeared, which transitioned to a fully photorealistic 50-million polygon rendering.

Nvidia_Project_Holodeck

Because Project Holodeck lets users interact with VR objects — and those objects to react to human touch — Koenigsegg was able to open the virtual Regera’s driver side door and step inside to see the interior details. He then took hold of the virtual steering wheel, “to show how Nvidia leverages VR PhysX technology so that your avatar’s hands don’t pass through virtual objects.”

Project Holodeck also let the demo viewers see “an exploded view of the car,” with a “cluster of disassembled parts in an animation that played out in real-time.”

VentureBeat notes that, “the new processor for artificial intelligence applications, the Tesla V100 [also called Volta] … has 21 billion transistors … an order of magnitude more powerful than the 15-billion transistor Pascal-based processor that Nvidia announced a year ago.” The 815 square millimeter chip, about the size of an Apple Watch face, has 5,120 CUDA processing cores, performs at 7.5 FP64 teraflops and can handle 120 Tensor teraflops. Samsung manufactures it for Nvidia.

Huang stated that, “deep learning neural network research started to pay off about five years ago, when researchers started using graphics processing units (GPUs) to process data in parallel to train neural networks quickly.” This year, Nvidia plans to train 100,000 developers to use deep learning.

In 2015 Microsoft’s deep learning program ResNet required 7 exaflops of processing power, Baidu’s Deep Speech 2 AI required 20 exaflops in 2016, and Google’s NMT required 105 exaflops in 2017. Microsoft is working on a new ResNet version that “chains 64 Volta chips together for processing.”