ExAvatar - A 3D digital avatar generation model launched by Meta

What is ExAvatar?

ExAvatar is a 3D digital avatar generation model jointly developed by DGIST and Meta's Codec Avatars Lab. It clones a person's image from a short video and transforms it into a 3D digital avatar. It supports rapid capture of full-body movements and expressions via mobile phone scanning, converting them into realistic 3D models. ExAvatar combines SMPL-X models and 3D Gaussian technology to achieve high-quality dynamic rendering, support full-body animation, reduce artifacts, and provide a natural and smooth experience. It is suitable for animation, games, and VR/AR fields, simplifying the 3D avatar creation process and broadening application scenarios.

ExAvatar's main functions

• Full-body 3D drivingSupports animation of full-body movements and expressions, including the body, hands, and face.
• Hybrid representationCombining 3D Gaussian and surface mesh technologies improves geometric and appearance consistency and reduces artifacts in animation.
• Convenient captureYou can quickly create a 3D avatar by scanning with your mobile phone; it's very easy to do.
• High-quality rendering: Utilize advanced algorithms to achieve high-quality rendering of dynamic performance and visual effects.
• Technical optimization: Optimize the expressiveness and naturalness of hands and faces through SMPL-X model and multilayer perceptron (MLP) processing.

ExAvatar's technical principles

• SMPL-X whole-body parametric mesh modelExAvatar uses the SMPL-X model to capture human poses and movements. SMPL-X is a highly parametric 3D human model that can accurately simulate various human postures.
• 3D Gaussian sputtering technology (3DGS)Used to improve the realism and efficiency of rendering, 3D models look more realistic by simulating the scattering of light on a surface.
• Hybrid representationExAvatar combines 3D Gaussian and surface meshes to ensure geometric and appearance consistency across different poses and reduce artifacts that may appear in new poses.
• Multilayer perceptron (MLP)It is used to process the features of each Gaussian, extract key information, and combine this information with a standard grid to form a 3D avatar that can be animated in standard space.
• Linear hybrid interpolation (LBS) algorithmUsed for animation processing, ExAvatar can generate smooth and natural transitions in this way.
• Joint offset and facial offsetBefore training ExAvatar, the research team performed joint registration of the body, hands, and face, and introduced these offsets to optimize the length of the hand bones and the shape of the facial region, thereby improving the expressiveness and naturalness of the avatar.
• Connection-based regularizerReduce artifacts in new expressions and poses by ensuring that 3D Gaussian meshes match the SMPL-X mesh topology through predefined connectivity.

ExAvatar's project address

Application scenarios of ExAvatar

• 3D animation productionAnimators can use ExAvatar to quickly create realistic 3D characters for use in the production of animated films, TV series, or short films.
• Game developmentGame developers can use ExAvatar technology to design dynamic full-body movements and expressions for game characters, enhancing the gaming experience.
• Virtual Reality (VR)In a VR environment, users can generate their own 3D avatar using ExAvatar for an immersive experience.
• Augmented Reality (AR)ExAvatar can be used in AR applications, such as adding interactivity to social media, advertising, or educational applications.
• Movie special effectsFilm special effects artists can use ExAvatar technology to capture actors' movements and expressions for later special effects production.