Holistic 3D Human and Scene Mesh Estimation from Single View Images

12/02/2020
by   Zhenzhen Weng, et al.
0

The 3D world limits the human body pose and the human body pose conveys information about the surrounding objects. Indeed, from a single image of a person placed in an indoor scene, we as humans are adept at resolving ambiguities of the human pose and room layout through our knowledge of the physical laws and prior perception of the plausible object and human poses. However, few computer vision models fully leverage this fact. In this work, we propose an end-to-end trainable model that perceives the 3D scene from a single RGB image, estimates the camera pose and the room layout, and reconstructs both human body and object meshes. By imposing a set of comprehensive and sophisticated losses on all aspects of the estimations, we show that our model outperforms existing human body mesh methods and indoor scene reconstruction methods. To the best of our knowledge, this is the first model that outputs both object and human predictions at the mesh level, and performs joint optimization on the scene and human poses.

READ FULL TEXT

page 4

page 8

page 12

page 13

research
02/27/2020

Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Image

Semantic reconstruction of indoor scenes refers to both scene understand...
research
03/13/2019

Putting Humans in a Scene: Learning Affordance in 3D Indoor Environments

Affordance modeling plays an important role in visual understanding. In ...
research
11/14/2018

No-Frills Human-Object Interaction Detection: Factorization, Appearance and Layout Encodings, and Training Techniques

We show that with an appropriate factorization, and encodings of layout ...
research
07/27/2023

Physically Plausible 3D Human-Scene Reconstruction from Monocular RGB Image using an Adversarial Learning Approach

Holistic 3D human-scene reconstruction is a crucial and emerging researc...
research
01/12/2023

Scene-Aware 3D Multi-Human Motion Capture from a Single Camera

In this work, we consider the problem of estimating the 3D position of m...
research
02/22/2020

Shallow2Deep: Indoor Scene Modeling by Single Image Understanding

Dense indoor scene modeling from 2D images has been bottlenecked due to ...

Please sign up or login with your details

Forgot password? Click here to reset