FLAG: Flow-based 3D Avatar Generation from Sparse Observations

by   Sadegh Aliakbarian, et al.

To represent people in mixed reality applications for collaboration and communication, we need to generate realistic and faithful avatar poses. However, the signal streams that can be applied for this task from head-mounted devices (HMDs) are typically limited to head pose and hand pose estimates. While these signals are valuable, they are an incomplete representation of the human body, making it challenging to generate a faithful full-body avatar. We address this challenge by developing a flow-based generative model of the 3D human body from sparse observations, wherein we learn not only a conditional distribution of 3D human pose, but also a probabilistic mapping from observations to the latent space from which we can generate a plausible pose along with uncertainty estimates for the joints. We show that our approach is not only a strong predictive model, but can also act as an efficient pose prior in different optimization settings where a good initial latent code plays a major role.


page 6

page 8


Task-Generic Hierarchical Human Motion Prior using VAEs

A deep generative model that describes human motions can benefit a wide ...

Adversarial Parametric Pose Prior

The Skinned Multi-Person Linear (SMPL) model can represent a human body ...

A Generative Model of People in Clothing

We present the first image-based generative model of people in clothing ...

HMD-NeMo: Online 3D Avatar Motion Generation From Sparse Observations

Generating both plausible and accurate full body avatar motion is the ke...

Driving-Signal Aware Full-Body Avatars

We present a learning-based method for building driving-signal aware ful...

CLOTH3D: Clothed 3D Humans

This work presents CLOTH3D, the first big scale synthetic dataset of 3D ...

JUMPS: Joints Upsampling Method for Pose Sequences

Human Pose Estimation is a low-level task useful for surveillance, human...

Please sign up or login with your details

Forgot password? Click here to reset