Learning to Reconstruct People in Clothing from a Single RGB Camera

by   Thiemo Alldieck, et al.

We present a learning-based model to infer the personalized 3D shape of people from a few frames (1-8) of a monocular video in which the person is moving, in less than 10 seconds with a reconstruction accuracy of 5mm. Our model learns to predict the parameters of a statistical body model and instance displacements that add clothing and hair to the shape. The model achieves fast and accurate predictions based on two key design choices. First, by predicting shape in a canonical T-pose space, the network learns to encode the images of the person into pose-invariant latent codes, where the information is fused. Second, based on the observation that feed-forward predictions are fast but do not always align with the input images, we predict using both, bottom-up and top-down streams (one per view) allowing information to flow in both directions. Learning relies only on synthetic 3D data. Once learned, the model can take a variable number of frames as input, and is able to reconstruct shapes even from a single image with an accuracy of 6mm. Results on 3 different datasets demonstrate the efficacy and accuracy of our approach.


page 1

page 7

page 8


Optimizable Object Reconstruction from a Single View

3D shape reconstruction from a single image is a highly ill-posed proble...

Learning Pose-invariant 3D Object Reconstruction from Single-view Images

Learning to reconstruct 3D shapes using 2D images is an active research ...

Vid2Actor: Free-viewpoint Animatable Person Synthesis from Video in the Wild

Given an "in-the-wild" video of a person, we reconstruct an animatable m...

Single-Camera 3D Head Fitting for Mixed Reality Clinical Applications

We address the problem of estimating the shape of a person's head, defin...

TetraTSDF: 3D human reconstruction from a single image with a tetrahedral outer shell

Recovering the 3D shape of a person from its 2D appearance is ill-posed ...

SiCloPe: Silhouette-Based Clothed People

We introduce a new silhouette-based representation for modeling clothed ...

Video Based Reconstruction of 3D People Models

This paper describes how to obtain accurate 3D body models and texture o...

Please sign up or login with your details

Forgot password? Click here to reset