October 14, 2016
In fall 2014, Google began developing a project it just unveiled: the Jump 3D video capture and production platform for virtual reality. Just as the newly released Daydream View makes the VR headset affordable, with Jump, the company hopes to make VR production both less expensive and, with 3D, more immersive. Although 3D has lost popularity, its use for virtual reality creates more of a “you are there” experience by emulating the way human vision works. Most 360-degree videos are currently still shot in 2D.
Variety notes that watching 2D video in a VR headset “feels like sitting in a round room without corners.” With Jump, VR producers can use 3D to add depth. “Before this, stereo 360 was entirely for high-end visual effects studios,” said Jump technical lead Sameer Agarwal.
Google realized that to achieve 3D, “they could simply arrange a set number of cameras in a circle, and have computers simulate an infinite number virtual cameras that would fill in to allow omni-directional 3D video capture,” a technique called Omnidirectional Stereo Projection.
The team determined that “it was actually fairly simple to build capture hardware with an array of 16 GoPro cameras, which happened to already have the optimal field-of-view.” They next built the software that, rather than stitching the images together, “instead interpolates imagery from those ‘virtual’ cameras’.”
With color correction and “other automated editing functionality,” the process is 4,000 times slower than real time, said Agarwal. To speed it up, Google moved it to the cloud, where “processing an hour of Jump footage with 1,000 cores in the cloud takes merely 10 hours,” with stitching being automated.
Producers can now shoot with the Jump rig, upload all 16 cameras’ footage to the cloud with a Mac application, and the Google servers turn it into a 3D VR video. Although the filmmaker may not have a lot of control over the process, it also “significantly lowers the barrier of entry for 3D video.”
The Jump camera prototype, dubbed Odyssey, was first shown at Google’s I/O developer conference in 2015, with the first rigs soon shipped to “select partners,” including YouTube production studios in Los Angeles, New York, London and Tokyo. Jump partners GoPro, The Atlantic and the AP “have since shot dozens of videos with the platform.”