LatentHuman: Shape-and-Pose Disentangled Latent Representation for Human Bodies

11/30/2021
by   Sandro Lombardi, et al.
0

3D representation and reconstruction of human bodies have been studied for a long time in computer vision. Traditional methods rely mostly on parametric statistical linear models, limiting the space of possible bodies to linear combinations. It is only recently that some approaches try to leverage neural implicit representations for human body modeling, and while demonstrating impressive results, they are either limited by representation capability or not physically meaningful and controllable. In this work, we propose a novel neural implicit representation for the human body, which is fully differentiable and optimizable with disentangled shape and pose latent spaces. Contrary to prior work, our representation is designed based on the kinematic model, which makes the representation controllable for tasks like pose animation, while simultaneously allowing the optimization of shape and pose for tasks like 3D fitting and pose tracking. Our model can be trained and fine-tuned directly on non-watertight raw data with well-designed losses. Experiments demonstrate the improved 3D reconstruction performance over SoTA approaches and show the applicability of our method to shape interpolation, model fitting, pose tracking, and motion retargeting.

READ FULL TEXT
research
04/01/2021

NPMs: Neural Parametric Models for 3D Deformable Shapes

Parametric 3D models have enabled a wide variety of tasks in computer gr...
research
05/14/2019

Learning 3D Human Body Embedding

Although human body shapes vary for different identities with different ...
research
01/20/2022

SPAMs: Structured Implicit Parametric Models

Parametric 3D models have formed a fundamental role in modeling deformab...
research
03/02/2022

H4D: Human 4D Modeling by Learning Neural Compositional Representation

Despite the impressive results achieved by deep learning based 3D recons...
research
08/05/2022

Disentangling 3D Attributes from a Single 2D Image: Human Pose, Shape and Garment

For visual manipulation tasks, we aim to represent image content with se...
research
09/12/2023

Learning Disentangled Avatars with Hybrid 3D Representations

Tremendous efforts have been made to learn animatable and photorealistic...
research
06/06/2023

Human 3D Avatar Modeling with Implicit Neural Representation: A Brief Survey

A human 3D avatar is one of the important elements in the metaverse, and...

Please sign up or login with your details

Forgot password? Click here to reset