Inferring Implicit 3D Representations from Human Figures on Pictorial Maps

08/30/2022
by   Raimund Schnürer, et al.
3

In this work, we present an automated workflow to bring human figures, one of the most frequently appearing entities on pictorial maps, to the third dimension. Our workflow is based on training data (i.e., SMPL-X, AGORA) and neural networks for single-view 3D reconstruction of real humans from photos. We first let a network consisting of fully connected layers estimate the depth coordinate of 2D pose points. The gained 3D pose points are inputted together with 2D masks of body parts into a deep implicit surface network to infer 3D signed distance fields (SDFs). By assembling all body parts, we derive 2D depth images and body part masks of the whole figure for different views, which are fed into a fully convolutional network to predict UV images. These UV images and the texture for the given perspective are inserted into a generative network to inpaint the textures for the other views. Finally, the generated textures are assigned to the inferred body parts in a ray marcher. We test our workflow with 12 pictorial human figures after having validated several network configurations. The created 3D models look generally promising, especially when considering the challenges of silhouette-based 3D recovery and real-time rendering of the implicit SDFs. Further improvement is needed to reduce gaps between the body parts and to add pictorial details to the textures. Overall, the constructed figures may be used for animation and storytelling in digital 3D maps.

READ FULL TEXT

page 9

page 14

page 16

page 17

page 18

page 23

page 25

research
11/09/2021

Pipeline for 3D reconstruction of the human body from AR/VR headset mounted egocentric cameras

In this paper, we propose a novel pipeline for the 3D reconstruction of ...
research
11/24/2021

EgoRenderer: Rendering Human Avatars from Egocentric Camera Images

We present EgoRenderer, a system for rendering full-body neural avatars ...
research
01/11/2021

Neural Re-Rendering of Humans from a Single Image

Human re-rendering from a single image is a starkly under-constrained pr...
research
04/09/2019

3DPeople: Modeling the Geometry of Dressed Humans

Recent advances in 3D human shape estimation build upon parametric repre...
research
09/23/2017

A Generic Regression Framework for Pose Recognition on Color and Depth Images

Cascaded regression method is a fast and accurate method on finding 2D p...
research
11/28/2018

Coordinate-based Texture Inpainting for Pose-Guided Image Generation

We present a new deep learning approach to pose-guided resynthesis of hu...
research
05/13/2022

Monocular Human Digitization via Implicit Re-projection Networks

We present an approach to generating 3D human models from images. The ke...

Please sign up or login with your details

Forgot password? Click here to reset