At SIGGRAPH 2015, Autodesk executives David Morin and Ben Guthrie described virtual production, its relationship with virtual reality and some newly released tools from their company to aid in the process. Virtual production began with Peter Jackson’s “Lord of the Rings,” got a bump of recognition with “Avatar,” and has been used on many films since. According to Morin and Guthrie, the process, which lets filmmakers create virtual worlds in-camera and composite CG and live action on set, is achieving momentum.

With the rise of digital VFX in feature films, the problem was that directors, cinematographers and actors had to imagine scenes with computer-generated images that were missing. Virtual production has put CG into the scene, making it easier to achieve better results with everything from framing to lighting.

With virtual production, the filmmaking process becomes a “collaborative and immersive content creation process that enables moviemakers to film virtual worlds in-camera.”

“It’s no longer a linear process,” explains Morin. “Now we start shooting in VFX and loop back to live action, and then do more previsualization and work that into the scene. It’s a two-way street model.”

Capture is a challenge when a production has to digitize hundreds or even thousands of characters and props. “Creating these digital assets from scratch can take a long time,” Guthrie says. A fast predictable system is now being offered with Autodesk’s Memento, a photogrammetry engine combined with HD 3D. Motion capture is the second way to capture digital assets.

Rendering in real-time is another challenge, and one option is to use a game engine, says Guthrie. Autodesk just bought Stingray, a 3D game engine and design visualization tool. Version 1 of Stingray, which can output to Oculus headsets was shown by Autodesk and will be available soon.

“There’s a lot of focus now on virtual reality,” says Morin. “VR is a new medium that will take its place along with TV and movies. Virtual reality is about production. But they both share immersion and the technology is the same. Our vision is that all filmmakers will be able to do virtual production, as easily as regular production.”