Driving-Signal Aware Full-Body Avatars

05/21/2021
by   Timur Bagautdinov, et al.
4

We present a learning-based method for building driving-signal aware full-body avatars. Our model is a conditional variational autoencoder that can be animated with incomplete driving signals, such as human pose and facial keypoints, and produces a high-quality representation of human geometry and view-dependent appearance. The core intuition behind our method is that better drivability and generalization can be achieved by disentangling the driving signals and remaining generative factors, which are not available during animation. To this end, we explicitly account for information deficiency in the driving signal by introducing a latent space that exclusively captures the remaining information, thus enabling the imputation of the missing factors required during full-body animation, while remaining faithful to the driving signal. We also propose a learnable localized compression for the driving signal which promotes better generalization, and helps minimize the influence of global chance-correlations often found in real datasets. For a given driving signal, the resulting variational model produces a compact space of uncertainty for missing factors that allows for an imputation strategy best suited to a particular application. We demonstrate the efficacy of our approach on the challenging problem of full-body animation for virtual telepresence with driving signals acquired from minimal sensors placed in the environment and mounted on a VR-headset.

READ FULL TEXT

page 1

page 6

page 9

page 10

page 11

page 13

page 14

page 15

research
06/07/2022

Garment Avatars: Realistic Cloth Driving using Pattern Registration

Virtual telepresence is the future of online communication. Clothing is ...
research
07/20/2022

Drivable Volumetric Avatars using Texel-Aligned Features

Photorealistic telepresence requires both high-fidelity body modeling an...
research
05/03/2022

DANBO: Disentangled Articulated Neural Body Representations via Graph Neural Networks

Deep learning greatly improved the realism of animatable human models by...
research
03/11/2022

FLAG: Flow-based 3D Avatar Generation from Sparse Observations

To represent people in mixed reality applications for collaboration and ...
research
12/01/2022

NeuWigs: A Neural Dynamic Model for Volumetric Hair Capture and Animation

The capture and animation of human hair are two of the major challenges ...
research
04/24/2023

Auto-CARD: Efficient and Robust Codec Avatar Driving for Real-time Mobile Telepresence

Real-time and robust photorealistic avatars for telepresence in AR/VR ha...
research
12/15/2022

HUM3DIL: Semi-supervised Multi-modal 3D Human Pose Estimation for Autonomous Driving

Autonomous driving is an exciting new industry, posing important researc...

Please sign up or login with your details

Forgot password? Click here to reset