Oscar-nominated “Pearl,” the short VR film directed by Passion Pictures’ Patrick Osborne, was the focus of a discussion about ambisonic audio during this week’s ETC conference on VR/AR in Las Vegas. Google Spotlight Stories creative director, audio and music Scot Stafford and Dramatic Audio Post sound designer/mixer Jamey Scott described their work on the film as well as their path to working on 360-degree and VR movies. Stafford pointed to his work at a story incubator at Motorola, run by Google, to make content for the mobile platform.
Stafford described how his experiences custom-building stories for smartphone VR (which he dubbed “little VR”) led to creating content for room-scale VR. At Google Spotlight Stories, he began to develop a wish list for audio tools.
“We started with interactive music and gaze-driven mixing, emitters,” he explained. “Then we made a big move with headlock stereo and ambisonic audio.” He pointed out some of the limitations of previous audio technology. “Probably 80 to 90 percent of the people we’ve worked with have spatialization concerns,” he said.
For “Pearl,” one of his first tasks was to find a custom song. Stafford said the company invited 15 songwriters to participate and early on zeroed in on “No Wrong Way Home,” by Alexis Harte (and co-writer JJ Wiesler). “The great thing about ambisonics is how naturally it rotates in 360 degree space,” said Stafford. “A nice ambisonic recording makes you feel like you’re really there. Does 6DOF [six degrees of freedom] kill the ambisonics? I don’t think so. Stereo won’t die, ambisonics won’t die.”
Scott described his career beginning in the early multimedia days of CD-ROMs, leading to a job in the early game company Presto Studios and some VR projects in the mid 1990s.
“That included spherical nodes you could pan which was a new technology, and faux object emitters,” he said. “We were creating an aural experience, which is what we are doing today in VR. That was my first work in creating audio for an immersive cinematic experience.”
Scott said he developed his own way to do ambisonic mixing in the context of a traditional film, on the movie “Help,” which was done at Warner Bros. Through trial and error, he discovered that mono panned sound better.
“On ‘Pearl,’ we ended up doing ambisonic mixing with hero tracks, and spent a lot of time on the ambient transitions,” he said. “We built it as a section of scenes that weren’t necessarily tied together, so that made creating the transitions a bit of a challenge. The artistic aesthetic is very much a marriage of game and film techniques, more technical from the game side and artistic from the film side.”
The audio team did multiple versions; the theatrical mix, in traditional 5.1 mix film format, was the one “gifted with the Oscar nomination.” Scott said his future goals for VR cinematic audio are a move away from ambisonics. “I’d like procedural acoustic simulations, convincing occlusion, easy to achieve custom HRTF for personalized experiences and resolving bottlenecks,” he said. “And what I hope to avoid is achieving total realism and being a technician first and an artist second.”