Learning Human Mesh Recovery in 3D Scenes

by   Zehong Shen, et al.

We present a novel method for recovering the absolute pose and shape of a human in a pre-scanned scene given a single image. Unlike previous methods that perform sceneaware mesh optimization, we propose to first estimate absolute position and dense scene contacts with a sparse 3D CNN, and later enhance a pretrained human mesh recovery network by cross-attention with the derived 3D scene cues. Joint learning on images and scene geometry enables our method to reduce the ambiguity caused by depth and occlusion, resulting in more reasonable global postures and contacts. Encoding scene-aware cues in the network also allows the proposed method to be optimization-free, and opens up the opportunity for real-time applications. The experiments show that the proposed network is capable of recovering accurate and physically-plausible meshes by a single forward pass and outperforms state-of-the-art methods in terms of both accuracy and speed.


page 1

page 5

page 8


NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video

We present a novel framework named NeuralRecon for real-time 3D scene re...

SMAP: Single-Shot Multi-Person Absolute 3D Pose Estimation

Recovering multi-person 3D poses with absolute scales from a single RGB ...

CenterHMR: a Bottom-up Single-shot Method for Multi-person 3D Mesh Recovery from a Single Image

In this paper, we propose a method to recover multi-person 3D mesh from ...

Dual Grid Net: hand mesh vertex regression from single depth maps

We present a method for recovering the dense 3D surface of the hand by r...

Distribution-Aligned Diffusion for Human Mesh Recovery

Recovering a 3D human mesh from a single RGB image is a challenging task...

Camera-Space Hand Mesh Recovery via Semantic Aggregation and Adaptive 2D-1D Registration

Recent years have witnessed significant progress in 3D hand mesh recover...

Rank-One Prior: Toward Real-Time Scene Recovery

Scene recovery is a fundamental imaging task for several practical appli...

Please sign up or login with your details

Forgot password? Click here to reset