What if you could restore a film — at 12 million frames per second — in two weeks? That’s a case study that Video Gorillas chief executive Jason Brahms described in detail at the HPA Tech Retreat, capping off how his company has spent the last ten years developing tools enabled by artificial intelligence. In 2007, Brahms, who was with Sony Pictures Imageworks, first met a developer pitching his facial recognition software. “I asked him if he had software that could compare different movie cuts and find differences between them,” he recalled.

Eleven years of R&D later, Video Gorillas is delivering real-world AI-enabled tools. “Our mission is to develop AI-powered solutions to decrease manual labor — not creative labor — in post production workflows and improve time-to-market for content owners,” explained Brahms.

The first product he showed was Bigfoot Frame Compare, which was a solution for the question Brahms posed in 2007. Bigfoot “enables people to look at versions of pictures and discover differences and similarities,” of director’s cuts, syndication masters and other cuts. “It eliminates human beings from arduous hours of doing a conform, and speeds up the process,” said Brahms.

According to Brahms, they realized early on that pixel matching methods wouldn’t support many M&E use cases. “So we looked at existing interest point matching methods SURF and SIFT, and initially got 84 to 92 percent accuracy,” he said. “But we realized that SURF and SIFT would not work at scale due to inefficient processing speeds and large index sizes.”

That led Video Gorillas to innovate a Frequency Domain Descriptor (FDD), in which an interest point and its surrounding are divided into subareas, with a FDD of each area created by applying a discrete Fourier transform (DFT). “Over the course of the next few years, we focused on various problems and, in 2011, did a proof of concept with Sony on ‘Designing Women’ with Sony Pictures Colorworks,” said Brahms.

In the past couple of years, he added, “the business has caught up to the technical capabilities,” with its 42 TV projects in 2017 jumping to 300 in 2018. Video Gorillas also was instrumental in the Netflix release of director Orson Welles’ last movie, “The Other Side of the Wind.”

“The objective was to conform the correct film scans to the reference picture, which was 3.5 hours of rough edits that Orson had made throughout the years, compiled in one long file with terrible quality,” said Brahms. Assets included 110 hours of film scan footage comprised of 12,000 files and multiple scans of many shots including 16mm, 35mm, B&W, color, 16mm-to-35mm blowups, dailies, circle takes, trims and work print. “There were 302,400 frames in the movie and 9.5 million frames in the scans,” noted Brahms.

The project required Brahms and his team to “dust off” an experimental GPU version of the software. “We compared over two trillion frames or 12 million frames per second in doing the frame compares,” he said. “We ingested the material in three days, did the analysis in four days and the human validation in nine days, completing the project in two weeks. It would have taken months if human beings had done it.”