“Digital Mark” Character
![NVIDIA Omniverse Audio2Face NVIDIA Omniverse Audio2Face](/content/dam/en-zz/Solutions/gtcf20/omniverse/audio2face/nvidia-omniverse-audio2face-128.png)
NVIDIA Audio2Face
Instantly create expressive facial animation from just an audio source using generative AI.
NVIDIA Audio2Face beta is a foundation application for animating 3D characters facial characteristics to match any voice-over track, whether for a game, film, real-time digital assistant, or just for fun. You can use the Universal Scene Description (OpenUSD)-based app for interactive real-time applications or as a traditional facial animation authoring tool. Run the results live or bake them out, it’s up to you.
Audio2Face is preloaded with “Digital Mark”— a 3D character model that can be animated with your audio track, so getting started is simple—just select your audio and upload. The audio input is then fed into a pre-trained Deep Neural Network and the output drives the 3D vertices of your character mesh to create the facial animation in real-time. You also have the option to edit various post-processing parameters to edit the performance of your character. The results you see on this page are mostly raw outputs from Audio2Face with little to no post-processing parameters edited.
Simply record a voice audio track, input into the app, and see your 3D face come alive. You can even generate facial animations live using a microphone.
Audio2Face will be able to process any language easily. And we’re continually updating with more and more languages.
Audio2Face lets you retarget to any 3D human or human-esque face, whether realistic or stylized. This makes swapping characters on the fly—whether human or animal—take just a few clicks.
It’s easy to run multiple instances of Audio2Face with as many characters in a scene as you like - all animated from the same, or different audio tracks. Breathe life and sound into dialogue between a duo, a sing-off between a trio, an in-sync quartet, and beyond. Plus, you can dial up or down the level of facial expression on each face and batch output multiple animation files from multiple audio sources.
Audio2Face gives you the ability to choose and animate your character’s emotions in the wink of an eye. The AI network automatically manipulates the face, eyes, mouth, tongue, and head motion to match your selected emotional range and customized level of intensity, or, automatically infers emotion directly from the audio clip.
Asset Credit: Blender Studio
The latest update to Audio2Face now enables blendshape conversion and also blendweight export options. Plus, the app now supports export-import with Blendshapes for Blender and Epic Games Unreal Engine to generate motion for characters using their respective Omniverse Connectors.
A2F has been tuned for emotional responses and features improved audio quality. It can now be run locally on the GPU.
To install Omniverse Audio2Face, follow the steps below:
NVIDIA Audio2Face is available today as a production microservice and can be accessed with the purchase of NVIDIA AI Enterprise for production. For early access or evaluation, please test out the 90 eval or use an API key from ai.nvidia.com