DeepAI AI Chat
Log In Sign Up

Pipeline for 3D reconstruction of the human body from AR/VR headset mounted egocentric cameras

11/09/2021
by   Shivam Grover, et al.
16

In this paper, we propose a novel pipeline for the 3D reconstruction of the full body from egocentric viewpoints. 3-D reconstruction of the human body from egocentric viewpoints is a challenging task as the view is skewed and the body parts farther from the cameras are occluded. One such example is the view from cameras installed below VR headsets. To achieve this task, we first make use of conditional GANs to translate the egocentric views to full body third-person views. This increases the comprehensibility of the image and caters to occlusions. The generated third-person view is further sent through the 3D reconstruction module that generates a 3D mesh of the body. We also train a network that can take the third person full-body view of the subject and generate the texture maps for applying on the mesh. The generated mesh has fairly realistic body proportions and is fully rigged allowing for further applications such as real-time animation and pose transfer in games. This approach can be key to a new domain of mobile human telepresence.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 6

page 7

page 9

page 10

03/13/2021

NeuralHumanFVV: Real-Time Neural Volumetric Human Performance Rendering using RGB Cameras

4D reconstruction and rendering of human activities is critical for imme...
08/30/2022

Inferring Implicit 3D Representations from Human Figures on Pictorial Maps

In this work, we present an automated workflow to bring human figures, o...
02/16/2020

PeelNet: Textured 3D reconstruction of human body using single view RGB image

Reconstructing human shape and pose from a single image is a challenging...
11/18/2022

UnconFuse: Avatar Reconstruction from Unconstrained Images

The report proposes an effective solution about 3D human body reconstruc...
04/11/2018

View Extrapolation of Human Body from a Single Image

We study how to synthesize novel views of human body from a single image...
11/30/2021

Robust 3D Garment Digitization from Monocular 2D Images for 3D Virtual Try-On Systems

In this paper, we develop a robust 3D garment digitization solution that...
05/16/2018

Predicting the Next Best View for 3D Mesh Refinement

3D reconstruction is a core task in many applications such as robot navi...