February 15, 2017
Microsoft has come up with a new camera rig that allows HoloLens mixed reality app makers to capture video from a HoloLens and make it easier to show a person interacting with that app, something Microsoft dubs “spectator view.” The details of the hardware-software combo were published as open source on the HoloLens’ GitHub page. The HoloLens headset is wireless, which lets the user move around the room freely, and is based on four cameras, lightly tinted lenses and a holographic processing unit.
The Verge calls the new camera set-up, “a small step, but it’s one headed in the right direction for head-mounted displays,” that comes at a time when VR/AR companies are “running into ethical boundaries as well, sometimes relying on special effects rather than ‘real’ video to get their points across.”
Showing off apps has become a challenge for HoloLens app developers, who contend with the small field of view that makes CGI images look tiny. Up until now, they have relied on the HoloLens’ built-in camera, which records video from the headset wearer’s point of view; the downsides are that the video quality is low and the outward-facing camera “can’t provide visuals of the person actually wearing the HoloLens.”
“Mixed reality capture is kind of the equivalent of the camera on your cell phone,” said Microsoft head of HoloLens business strategy Ben Reed. “It’s handy, it’s convenient, it does a good job of what it’s meant to do, but it’s not designed to be broadcast quality, what you’d see on TV. We realized we needed another way show other people what the wearer is seeing.”
Microsoft’s own solution is an in-house rig that uses a RED Dragon camera, way out of the price range of most app developers.
This new “spectator view” solution simply requires any camera with an HDMI output. The user mounts the HoloLens to the camera, stabilized on a tripod (which requires a custom-made mount), and then wirelessly connects it to a PC running Unity, to share positioning data. Next step, the user outputs video via an HDMI cable, sending positioning data and video to the PC. Unity takes care of processing the data, and the result is “video of people walking around the room wearing HoloLens headsets” as well as “the apps and the digital objects and the games they’re playing around them.”
Reed notes that HoloLens customers have been requesting this new tool, although “it’s still unclear exactly how many HoloLens users are out there in the world,” with Microsoft saying only that sales are “in thousands.” Although the new rig may not impact that many people, says The Verge, in the “rapidly advancing world of augmented reality, virtual reality, and mixed reality, it at least offers a hack for making videos that feel, well, real.”
The Microsoft developer site provides extensive details regarding the spectator view approach.