A technique called light field capture will become the foundation for photoreal virtual actors for virtual reality, says Paul Debevec, chief visual officer at the University of Southern California’s Institute of Creative Technologies (ICT). At the recent VRLA Expo, Debevec gave a talk on the topic that explored two decades of research and development in light field capture technology, and described the basics of what makes this technique so compelling to create photorealistic virtual reality.

In light field capture, a photograph is a record of the rays of light that make up what the lens sees, explains Turnstyle News. “When multiple images are taken at different angles it is possible to know where two rays of light intersect, and thus create a kind of virtual camera.”

Debevec and his ICT colleagues demonstrated light field capture in a video for OTOY, a cloud-rendering company in Los Angeles. With a camera, a fisheye lens, a motorized rig and a proprietary software, he turned an office into a full stereoscopic image.

“Now when you move your head around in VR — whether it’s OTOY’s offices or one of the other scenes that have been recorded — you get the corresponding perspective shifts of the scene that’s there,” said Debevec. “You can even push your head outside of the viewing volume a little bit — forward and back — and still reconstruct views because the cameras that were around you still saw all the rays that you need.”

This technique can resolve some of the issues, such as stitch lines, that prevent VR from being completely believable, the Holy Grail for VR creators. Up until now, creators have used real actors to play the characters in pre-rendered performances, but it’s now become possible to render them in real time, as Industrial Light & Magic’s xLAB has demonstrated.

Debevec and other VR experts believe that we’ll begin to see photoreal VR characters with “some form of artificial intelligence,” enabling realistic motor skills and narrative choices. “Actors don’t have anything to worry about yet,” says Debevec, “and probably not realistically for maybe 15 or 20 years.”