September 2, 2016
After buying Oculus in 2014, Facebook furthered its ambitions in virtual reality last fall by creating a computational photography team made up of former Microsoft experts. They were tasked with creating photo and video features that bring virtual reality to the greater public. One of the first projects was to enable 360-degree photos on Facebook. Now, Facebook is offering automated video stabilization for 360-degree videos shot by consumer-level cameras, a big step in avoiding the nausea that VR can induce in some people.
Recode quotes Rick Szeliski, head of the computational photography team, as saying that, “360-degree video stabilization makes all the difference… If you put on a VR headset and the video’s wobbling around, you get motion-sick in just a few seconds.”
The team is also at work on a 3D-photo software for smartphones, as a way of bringing VR creation to widely available devices. “I think we could write an app today that is very similar to iPhone panoramas, that captures a bit more information and creates a left eye-right eye [stereoscopy experience],” said team member Matt Uyttendaele.
Wired stresses the importance of the new software, calling it one of the “great unsolved issues of 360-degree video: the vomit problem.” Johannes Kopf headed up a team of engineers that spent a year designing the algorithm now being tested on Facebook, reports Wired.
“A shaky skiing video will look like a smooth slide down the mountain,” said Kopf, who describes the method as “deformed rotation,” which “can stabilize a 360 video in less than 22 milliseconds per frame.” Kopf also notes that the algorithm can create “hyperlapse” 360-degree videos, which compresses footage into shorter run times.
The algorithm is also “separate from the work Facebook is doing with its open source Surround 360 camera,” a professional-level camera.
“Facebook is building all of the tools needed for VR, from Oculus Rift headsets to $30,000 360-degree cameras,” notes Recode. “So it makes sense that consumer creation tools would be on its radar, too.” The company is “always thinking years into the future” which gives the computational photography team “that same luxury.”
“We’re not just gilding the lily, we’re not just making small improvements,” Szeliski said. “We work on the big, high-impact things.”