Log In Sign Up

Camera-Space Hand Mesh Recovery via Semantic Aggregation and Adaptive 2D-1D Registration

by   Xingyu Chen, et al.

Recent years have witnessed significant progress in 3D hand mesh recovery. Nevertheless, because of the intrinsic 2D-to-3D ambiguity, recovering camera-space 3D information from a single RGB image remains challenging. To tackle this problem, we divide camera-space mesh recovery into two sub-tasks, i.e., root-relative mesh recovery and root recovery. First, joint landmarks and silhouette are extracted from a single input image to provide 2D cues for the 3D tasks. In the root-relative mesh recovery task, we exploit semantic relations among joints to generate a 3D mesh from the extracted 2D cues. Such generated 3D mesh coordinates are expressed relative to a root position, i.e., wrist of the hand. In the root recovery task, the root position is registered to the camera space by aligning the generated 3D mesh back to 2D cues, thereby completing camera-space 3D mesh recovery. Our pipeline is novel in that (1) it explicitly makes use of known semantic relations among joints and (2) it exploits 1D projections of the silhouette and mesh to achieve robust registration. Extensive experiments on popular datasets such as FreiHAND, RHD, and Human3.6M demonstrate that our approach achieves state-of-the-art performance on both root-relative mesh recovery and root recovery. Our code is publicly available at


page 1

page 3

page 6

page 11

page 13

page 14

page 15


End-to-end Hand Mesh Recovery from a Monocular RGB Image

In this paper, we present a HAnd Mesh Recovery (HAMR) framework to tackl...

Im2Mesh GAN: Accurate 3D Hand Mesh Recovery from a Single RGB Image

This work addresses hand mesh recovery from a single RGB image. In contr...

Deep Monocular 3D Human Pose Estimation via Cascaded Dimension-Lifting

The 3D pose estimation from a single image is a challenging problem due ...

PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images

We present PyMAF-X, a regression-based approach to recovering a full-bod...

Probabilistic Modeling for Human Mesh Recovery

This paper focuses on the problem of 3D human reconstruction from 2D evi...

HandTailor: Towards High-Precision Monocular 3D Hand Recovery

3D hand pose estimation and shape recovery are challenging tasks in comp...

3DCaricShop: A Dataset and A Baseline Method for Single-view 3D Caricature Face Reconstruction

Caricature is an artistic representation that deliberately exaggerates t...

Code Repositories