X-Avatar: Expressive Human Avatars

03/08/2023
by   Kaiyue Shen, et al.
0

We present X-Avatar, a novel avatar model that captures the full expressiveness of digital humans to bring about life-like experiences in telepresence, AR/VR and beyond. Our method models bodies, hands, facial expressions and appearance in a holistic fashion and can be learned from either full 3D scans or RGB-D data. To achieve this, we propose a part-aware learned forward skinning module that can be driven by the parameter space of SMPL-X, allowing for expressive animation of X-Avatars. To efficiently learn the neural shape and deformation fields, we propose novel part-aware sampling and initialization strategies. This leads to higher fidelity results, especially for smaller body parts while maintaining efficient training despite increased number of articulated bones. To capture the appearance of the avatar with high-frequency details, we extend the geometry and deformation fields with a texture network that is conditioned on pose, facial expression, geometry and the normals of the deformed surface. We show experimentally that our method outperforms strong baselines in both data domains both quantitatively and qualitatively on the animation task. To facilitate future research on expressive avatars we contribute a new dataset, called X-Humans, containing 233 sequences of high-quality textured scans from 20 participants, totalling 35,500 data frames.

READ FULL TEXT

page 1

page 7

page 8

research
10/11/2022

HiFECap: Monocular High-Fidelity and Expressive Capture of Human Performances

Monocular 3D human performance capture is indispensable for many applica...
research
10/10/2020

Learning a model of facial shape and expression from 4D scans

The field of 3D face modeling has a large gap between high-end and low-e...
research
12/02/2021

Neural Head Avatars from Monocular RGB Videos

We present Neural Head Avatars, a novel neural representation that expli...
research
05/03/2023

AG3D: Learning to Generate 3D Avatars from 2D Image Collections

While progress in 2D generative models of human appearance has been rapi...
research
08/10/2018

DeepWrinkles: Accurate and Realistic Clothing Modeling

We present a novel method to generate accurate and realistic clothing de...
research
03/29/2023

4D Facial Expression Diffusion Model

Facial expression generation is one of the most challenging and long-sou...
research
05/08/2023

HACK: Learning a Parametric Head and Neck Model for High-fidelity Animation

Significant advancements have been made in developing parametric models ...

Please sign up or login with your details

Forgot password? Click here to reset