As part of what it calls “production microservices,” Nvidia is adding an Avatar Cloud Engine (ACE) that lets game developers, as well as those who make tools and middleware, to integrate generative AI models into the digital avatars created for games and applications. The new ACE microservices “let developers build interactive avatars using AI models such as Nvidia Omniverse Audio2Face (A2F), which creates expressive facial animations from audio sources, and Nvidia Riva automatic speech recognition (ASR), for building customizable multilingual speech and translation applications using generative AI,” Nvidia says. Continue reading CES: Nvidia Avatar Cloud Engine Uses AI for Digital Characters

Epic Online Services, the development hub for Epic Games, has expanded its crossplay overlay initiative to include initial support for games on Microsoft Xbox, Nintendo Switch and Sony PlayStation, optimizing its free SDKs for multiplayer crossplay and making it easier for developers to link their games and communities for those platforms across multiple stores. Last summer, EOS launched a PC crossplay overlay. The current upgrade enhances “capabilities across all supported platforms,” the company says, noting “crossplay enables bigger games, bigger audiences” and global games industry growth by connecting more players with a single overlay. Continue reading Epic Offers Crossplay Support for Xbox, PlayStation, Switch

Films shot using the 3D world-building tool Mona will compete in what is being billed as “the world’s first metaverse short film festival.” Running September 26-29, the Mona Shorts Fest will take place in the immersive environment app that CEO Justin Melillo has coined “the Monaverse.” “With an entire film studio at your fingertips through Mona’s SDK and in-world experience, imagine what story you could tell,” the festival’s website suggests. Last summer, the company announced more than $14 million in Series A funds after “thousands of creators” used the platform to build experiences for Web3. Continue reading Films Shot ‘In-World’ at Mona Get Their Own Shorts Festival

Sony Electronics is launching its Mocopi mobile motion capture system in the United States. Using a dedicated smartphone app for iOS and Android, the wireless system enables full-body motion tracking, captured by six small, lightweight sensors. Sony has been marketing Mocopi in Japan where virtual streamers (also called “VTubers”) have been using the system to drive avatars and fictional animated characters. Mocopi allows users to go mobile with virtual reality, loosening time and location constraints. Sony is now taking preorders for the $499 Mocopi system, which ships July 14. Continue reading Sony Offers Affordable Phone-Based MoCap System in U.S.

The new Catalyst Stage at ESPN headquarters in Bristol, Connecticut blends physical space with virtual production and an LED wall. The facility “opens new possibilities in the future of content customization and scalability” for live, multi-camera shooting that includes augmented reality and extended reality in 1080p or UHD, with 4K available for specialty work, including advertising. ESPN calls Catalyst “the first studio of its kind to support both live multi-camera productions and cinematic-quality projects. GhostFrame, Unreal Engine, Disguise XR, Pixotope and Mark Roberts Motion Control cameras are among Catalyst’s tools. Continue reading New Catalyst Stage Brings Virtual Production to ESPN Studio

Epic Games is releasing MetaHuman Animator, which lets developers create nuanced facial animation by capturing an actor’s performance using an iPhone or stereo head-mounted camera system and a PC. The system eliminates the need for manual touch-ups, according to Epic, capturing “every subtle expression, look, and emotion” and replicating it onto a digital character for a faster performance capture workflow that allows more creative control. The new feature set uses a 4D solver to combine video and depth data with a MetaHuman representation of the performer. The animation is produced locally using GPU hardware, providing final results in “minutes.” Continue reading New Tool from Epic Simplifies High-Fidelity Facial Animation

Entertainment and communications leaders explored the impact of artificial intelligence and anticipated the larger changes ahead at the inaugural Synthetic Media Summit, presented by the Entertainment Technology Center at USC in partnership with NAB Amplify, SMPTE and sponsor Wizeline, in conjunction with the USC School of Cinematic Arts and USC Annenberg School for Communication and Journalism. Speakers addressed how new technology will make VFX cheaper and faster for studios, while for indies it will open new frontiers. Teaming AI with tools such as Unreal Engine is expected to level the playing field and launch a new era of virtual production. Continue reading Thought Leaders Analyze AI at ETC Synthetic Media Summit

Digital filmmaking tools have become increasingly accessible, and now Wonder Dynamics wants to make photorealistic CG characters available for any budget. The software firm says its product enables users to drag and drop computer-generated characters into live-action scenes as if they were custom generated. The company’s web-based editor, Wonder Studio, is billed as a full-blown tool, not a toy, and aims to help ease artists’ workload. The three-year-old startup has raised $12.5 million to date. Co-founders Nikola Todorovic, a writer-director, and actor Tye Sheridan, who starred in Steven Spielberg’s “Ready Player One,” say it’s the tool they’ve craved. Continue reading Wonder Dynamics Leverages AI for Web-Based CGI Platform

Virtual production in its current form has been around for about two years, but they’ve been light years. From novice to veteran, everyone is still learning the ins and outs, the difference between previs and techvis, 3D and 2.5D. The pandemic fast-forwarded “to about six months a development process that used to happen on a four-year cycle, so a lot of buzzwords and concepts went from zero to 100 percent very quickly,” Lux Machina president Zach Alexander shared at the first annual SVG Silicon Valley Video Summit (SVVS), which brought together more than 400 digital practitioners from numerous areas. Continue reading Panelists Evaluate Challenges, Benefits of Virtual Production

Connection, collaboration, and cooperation are three words that underscore almost everything we saw during four days at CES 2023. We anticipated this would not be a show of breakthrough innovations. Instead, we expected innovative ways to use recent advances. In broad categories, AR, haptics, and AI were much in evidence. Interesting light field displays and curved screens caught our eye. There were fewer cars but way more commercial vehicles and components driving “software-defined mobility.” TVs were secondary to connected ecosystems in Samsung and LG’s displays, while creators took center stage for Sony, Canon, and Nikon. Clear across the show, innovation may come from startups but to scale it takes giants. Continue reading CES: Show Floor Reveals the Ups and Downs of Tech Trends

Engineered for game design and other 3D workflows, the Asus Vivobook Pro 16X 3D OLED offers glasses-free spatial imaging. To do this, the laptop leverages something Asus calls Spatial Vision, an autostereoscopic OLED technology that creates a set of images for each eye using real-time eye tracking tech. With the 3D mode activated, the images are then projected through a microscopic layer of lenticular lenses built into the display panel. The technology is similar to the SpatialLabs technology Acer debuted in 2021, with one big difference: a Spatial Vision screen will appear 3D to multiple onlookers simultaneously. Continue reading CES: Asus Spatial Vision Brings Glasses-Free 3D to Laptops

The industry is buzzing about a new Sony product called Mocopi that offers motion capture priced for consumers. The Meta Quest-compatible Mocopi utilizes six tracking bands to be worn on the head, back, hands and feet. Priced at 49,500 yen (or about $358), Sony announced Mocopi on its Japanese YouTube channel, with a U.S. release expected toward the end of January 2023. Touted as a way to operate avatars in the metaverse or make videos, Mocopi will have an SDK that can import motion data into 3D animations. Continue reading Sony Targets the Metaverse with Consumer Motion Capture

Meta Platforms unveiled its anticipated mixed reality headset yesterday at Meta Connect. The $1,500 Quest Pro, which CEO Mark Zuckerberg introduced as “the first in our new line of advanced headsets,” becomes the first full-color AR/VR device to hit the U.S. market on October 25. Although Meta is positioning the Quest Pro as a productivity device, the thrust was that app makers will be encouraged to develop for it. Meta also announced that NBCUniversal content and Microsoft Office, Windows, Teams and Xbox Cloud Gaming are coming to Meta Quest headsets. Continue reading AR/VR Meta Quest Pro Targets Productivity Sector for $1,500

Virtual character developer platform Inworld AI has raised $50 million in a Series A funding round led by Section 32 and Intel Capital. The Mountain View-based startup — one of six companies chosen to participate in the 2022 Disney Accelerator — will create virtual characters for games, the metaverse and other entertainment and marketing applications. Because it is focused on providing an interior life, or “mind,” Inworld AI is platform agnostic, with APIs that work across Unity, Unreal Engine, Omniverse and others. Another convenient feature: it lets developers build characters by describing them in natural language. Continue reading Inworld Raises $50M to Create AI-Powered Virtual Characters

Production in the cloud, virtual workflows and remote work solutions were hot topics at the 2022 NAB Show in Las Vegas last month. Erik Weaver, director of adaptive production at the Entertainment Technology Center@USC, discussed new tools and techniques implemented during the production of ETC’s “Fathead,” a short film that experimented with in-camera real-time VFX and LED stages. Weaver joined “Fathead” virtual production producer Tom Thudiyanplackal, the short’s post-production supervisor James Blevins, and the global lead for Film/TV production partners at Amazon Web Services (AWS) Jack Wenzinger for a compelling NAB panel discussion, “ICVFX and the Cloud.” Continue reading ‘Fathead’ Team Explores Virtual Production During NAB Panel