H4D: Human 4D Modeling by Learning Neural Compositional Representation

03/02/2022
by   Boyan Jiang, et al.
8

Despite the impressive results achieved by deep learning based 3D reconstruction, the techniques of directly learning to model the 4D human captures with detailed geometry have been less studied. This work presents a novel framework that can effectively learn a compact and compositional representation for dynamic human by exploiting the human body prior from the widely-used SMPL parametric model. Particularly, our representation, named H4D, represents dynamic 3D human over a temporal span into the latent spaces encoding shape, initial pose, motion and auxiliary information. A simple yet effective linear motion model is proposed to provide a rough and regularized motion estimation, followed by per-frame compensation for pose and geometry details with the residual encoded in the auxiliary code. Technically, we introduce novel GRU-based architectures to facilitate learning and improve the representation capability. Extensive experiments demonstrate our method is not only efficacy in recovering dynamic human with accurate motion and detailed geometry, but also amenable to various 4D human related tasks, including motion retargeting, motion completion and future prediction.

READ FULL TEXT

page 5

page 15

page 17

page 18

page 19

page 20

page 21

page 22

research
03/15/2021

Learning Compositional Representation for 4D Captures with Neural ODE

Learning based representation has become the key to the success of many ...
research
11/30/2021

LatentHuman: Shape-and-Pose Disentangled Latent Representation for Human Bodies

3D representation and reconstruction of human bodies have been studied f...
research
08/09/2020

3D Human Motion Estimation via Motion Compression and Refinement

We develop a technique for generating smooth and accurate 3D human pose ...
research
08/18/2022

LoRD: Local 4D Implicit Representation for High-Fidelity Dynamic Human Modeling

Recent progress in 4D implicit representation focuses on globally contro...
research
01/31/2018

Robust 3D Human Motion Reconstruction Via Dynamic Template Construction

In multi-view human body capture systems, the recovered 3D geometry or e...
research
03/28/2022

Structured Local Radiance Fields for Human Avatar Modeling

It is extremely challenging to create an animatable clothed human avatar...
research
01/09/2017

MonoCap: Monocular Human Motion Capture using a CNN Coupled with a Geometric Prior

Recovering 3D full-body human pose is a challenging problem with many ap...

Please sign up or login with your details

Forgot password? Click here to reset