Production in the cloud, virtual workflows and remote work solutions were hot topics at the 2022 NAB Show in Las Vegas last month. Erik Weaver, director of adaptive production at the Entertainment Technology Center@USC, discussed new tools and techniques implemented during the production of ETC’s “Fathead,” a short film that experimented with in-camera real-time VFX and LED stages. Weaver joined “Fathead” virtual production producer Tom Thudiyanplackal, the short’s post-production supervisor James Blevins, and the global lead for Film/TV production partners at Amazon Web Services (AWS) Jack Wenzinger for a compelling NAB panel discussion, “ICVFX and the Cloud.”
Film and television production is becoming a collaborative effort among studios, equipment manufacturers and service providers. The production of “Fathead” helped illustrate how that collaborative process can be efficiently planned and executed.
“Virtual production is about more than just the LED wall. It is a whole suite of tools,” explained Thudiyanplackal, who participated in three NAB sessions focused on ETC’s action short, directed by C. Craig Patterson.
Left to right: Erik Weaver, Tom Thudiyanplackal, James Blevins, Jack Wenzinger
at the “ICVFX and the Cloud” panel (photo courtesy of Eric Rigney)
Wenzinger vividly illustrated that point during the “ICVFX and the Cloud” discussion. AWS Partners were brought to the project, like Arch Platform Technologies, whose enterprise-level SaaS dashboard helped the team spin-up a studio in the cloud, build and manage workstations, render farms and access storage and servers.
“The cloud has a natural place in optimizing remote collaboration, GPU, and storage usage supporting virtual production in all three phases: preproduction, on-set collaboration, and post,” Wenzinger said. “AWS provides a strategic outrigger to virtual production.”
High-capacity connectivity between production and the cloud is critical and there are several solutions available to connect to the cloud. “We were writing camera files in real-time from the ARRI Alexa Mini LF directly to the cloud at 800GB per second,” Weaver said. Included in the extensive toolkit, collaboration software from 5th Kind helped organize permissions, delegation and asset management.
A key aspect of virtual production is the emphasis on pre-production for the volume stage, where LED panel-lined walls supply the computer-generated environments and visual effects for what is essentially a live, onset shooting experience.
Bluescape — a virtual workspace for concept development through production — was “used to put all the components together in advance,” Weaver noted. Epic Games’ RealityCapture was essential for photogrammetry, translating 2D photographs into textured, 3D environments and objects, a “resource-intensive process” that Thudiyanplackal said allowed the scalability of cloud computing to shine.
A markerless, suitless motion capture system provided by Move.ai was tested on “Fathead.” Requiring 4 to 8 consumer-level action cameras placed in a circular arrangement to capture anywhere from 1 to 5 actors, the footage was uploaded to the cloud-based server where AI algorithms identified each actor to generate separate animation data streams.
Epic’s MetaHuman Creator was used to create digital actors, while the company’s Unreal Engine was used to assemble the 3D worlds that played in real-time on the LED walls, the backdrop for in-camera VFX.
Once the footage left the camera for the cloud, that is where the remainder of workflow took place. “We’re color grading on a DaVinci Resolve 18 in the cloud. We’re the first to do that and describe the process as on-set-remote color editorial,” Weaver said of the 4.5K Open Gate ARRIRAW 12-bit footage being streamed as a 10-bit 444 HEVC signal.
“World building will engender new content that will be produced and stay in the digital world,” said Blevins. “Micro studios will form organically from storytellers and production designers collaborating in a 360-degree environment. They will be playing a new game called ‘Let’s make a movie.’ The result will power the next generation of content.”
A white paper on “The Role of the Cloud in Virtual Production on ‘Fathead’” will be released soon, with a comprehensive analysis to follow.