Photorealistic Monocular 3D Reconstruction of Humans Wearing Clothing

04/19/2022
by   Thiemo Alldieck, et al.
0

We present PHORHUM, a novel, end-to-end trainable, deep neural network methodology for photorealistic 3D human reconstruction given just a monocular RGB image. Our pixel-aligned method estimates detailed 3D geometry and, for the first time, the unshaded surface color together with the scene illumination. Observing that 3D supervision alone is not sufficient for high fidelity color reconstruction, we introduce patch-based rendering losses that enable reliable color reconstruction on visible parts of the human, and detailed and plausible color estimation for the non-visible parts. Moreover, our method specifically addresses methodological and practical limitations of prior work in terms of representing geometry, albedo, and illumination effects, in an end-to-end model where factors can be effectively disentangled. In extensive experiments, we demonstrate the versatility and robustness of our approach. Our state-of-the-art results validate the method qualitatively and for different metrics, for both geometric and color reconstruction.

READ FULL TEXT

page 4

page 7

page 8

page 10

page 11

page 12

research
09/30/2022

MonoNeuralFusion: Online Monocular Neural 3D Reconstruction with Geometric Priors

High-fidelity 3D scene reconstruction from monocular videos continues to...
research
04/08/2020

ARCH: Animatable Reconstruction of Clothed Humans

In this paper, we propose ARCH (Animatable Reconstruction of Clothed Hum...
research
08/17/2021

ARCH++: Animation-Ready Clothed Human Reconstruction Revisited

We present ARCH++, an image-based method to reconstruct 3D avatars with ...
research
11/12/2022

Illumination-Based Color Reconstruction for the Dynamic Vision Sensor

This work demonstrates a novel, state of the art method to reconstruct c...
research
12/14/2022

PhoMoH: Implicit Photorealistic 3D Models of Human Heads

We present PhoMoH, a neural network methodology to construct generative ...
research
04/04/2019

3D Face Reconstruction Using Color Photometric Stereo with Uncalibrated Near Point Lights

We present a new color photometric stereo (CPS) method that can recover ...
research
04/24/2023

Learning Visibility Field for Detailed 3D Human Reconstruction and Relighting

Detailed 3D reconstruction and photo-realistic relighting of digital hum...

Please sign up or login with your details

Forgot password? Click here to reset