Nvidia has made its Audio2Face open source, a potential boon for game developers and other 3D uses such as customer service. The generative AI facial animation system brings lifelike speech and expression to avatars on an accelerated basis using real-time facial animation and lip-sync. It works by analyzing acoustic features to create a stream of animation data that is then mapped onto a character’s facial poses. The data translates to “accurate lip-sync and emotional expressions,” says Nvidia, noting the imagery can be rendered offline for pre-scripted content or streamed in real time for dynamic characters with accurate lip-sync and emotional expressions. Continue reading Nvidia Audio2Face AI Avatar-Generator Is Now Open Source

Microsoft has developed VASA, a framework for generating lifelike virtual characters with vocal capabilities including speaking and singing. The premiere model, VASA-1, can perform the feat in real time from a single static image and a vocalization clip. The research demo showcases realistic audio-enhanced faces that can be fine-tuned to look in different directions or change expression in video clips of up to one minute at 512 x 512 pixels and up to 40fps “with negligible starting latency,” according to Microsoft, which says “it paves the way for real-time engagements with lifelike avatars that emulate human conversational behaviors.” Continue reading Microsoft’s VASA-1 Can Generate Talking Faces in Real Time

Nvidia founder and CEO Jensen Huang shared his vision for a computer graphics industry transformed by AI, the metaverse and digital humans. “The combination of AI and computer graphics will power the metaverse, the next evolution of the Internet,” Huang told attendees at SIGGRAPH 2022 in Vancouver. To support this transformation, Nvidia unveiled the Avatar Cloud Engine (ACE) and discussed plans to build out the Universal Scene Description (USD) industry standard, which Huang called “the language of the metaverse.” New extensions for Omniverse and graphics workflow optimizations using machine learning were also part of the mix.
Continue reading Nvidia Unveils New Tools for AI, the Metaverse at SIGGRAPH