DeepAI AI Chat
Log In Sign Up

Pipeline for 3D reconstruction of the human body from AR/VR headset mounted egocentric cameras

by   Shivam Grover, et al.

In this paper, we propose a novel pipeline for the 3D reconstruction of the full body from egocentric viewpoints. 3-D reconstruction of the human body from egocentric viewpoints is a challenging task as the view is skewed and the body parts farther from the cameras are occluded. One such example is the view from cameras installed below VR headsets. To achieve this task, we first make use of conditional GANs to translate the egocentric views to full body third-person views. This increases the comprehensibility of the image and caters to occlusions. The generated third-person view is further sent through the 3D reconstruction module that generates a 3D mesh of the body. We also train a network that can take the third person full-body view of the subject and generate the texture maps for applying on the mesh. The generated mesh has fairly realistic body proportions and is fully rigged allowing for further applications such as real-time animation and pose transfer in games. This approach can be key to a new domain of mobile human telepresence.


page 1

page 2

page 3

page 4

page 6

page 7

page 9

page 10


NeuralHumanFVV: Real-Time Neural Volumetric Human Performance Rendering using RGB Cameras

4D reconstruction and rendering of human activities is critical for imme...

Inferring Implicit 3D Representations from Human Figures on Pictorial Maps

In this work, we present an automated workflow to bring human figures, o...

PeelNet: Textured 3D reconstruction of human body using single view RGB image

Reconstructing human shape and pose from a single image is a challenging...

UnconFuse: Avatar Reconstruction from Unconstrained Images

The report proposes an effective solution about 3D human body reconstruc...

View Extrapolation of Human Body from a Single Image

We study how to synthesize novel views of human body from a single image...

Robust 3D Garment Digitization from Monocular 2D Images for 3D Virtual Try-On Systems

In this paper, we develop a robust 3D garment digitization solution that...

Predicting the Next Best View for 3D Mesh Refinement

3D reconstruction is a core task in many applications such as robot navi...