BiHand: Recovering Hand Mesh with Multi-stage Bisected Hourglass Networks

08/12/2020
by   Lixin Yang, et al.
1

3D hand estimation has been a long-standing research topic in computer vision. A recent trend aims not only to estimate the 3D hand joint locations but also to recover the mesh model. However, achieving those goals from a single RGB image remains challenging. In this paper, we introduce an end-to-end learnable model, BiHand, which consists of three cascaded stages, namely 2D seeding stage, 3D lifting stage, and mesh generation stage. At the output of BiHand, the full hand mesh will be recovered using the joint rotations and shape parameters predicted from the network. Inside each stage, BiHand adopts a novel bisecting design which allows the networks to encapsulate two closely related information (e.g. 2D keypoints and silhouette in 2D seeding stage, 3D joints, and depth map in 3D lifting stage, joint rotations and shape parameters in the mesh generation stage) in a single forward pass. As the information represents different geometry or structure details, bisecting the data flow can facilitate optimization and increase robustness. For quantitative evaluation, we conduct experiments on two public benchmarks, namely the Rendered Hand Dataset (RHD) and the Stereo Hand Pose Tracking Benchmark (STB). Extensive experiments show that our model can achieve superior accuracy in comparison with state-of-the-art methods, and can produce appealing 3D hand meshes in several severe conditions.

READ FULL TEXT

page 1

page 2

page 4

page 10

research
02/25/2019

End-to-end Hand Mesh Recovery from a Monocular RGB Image

In this paper, we present a HAnd Mesh Recovery (HAMR) framework to tackl...
research
07/24/2019

Dual Grid Net: hand mesh vertex regression from single depth maps

We present a method for recovering the dense 3D surface of the hand by r...
research
09/03/2021

Towards Accurate Alignment in Real-time 3D Hand-Mesh Reconstruction

3D hand-mesh reconstruction from RGB images facilitates many application...
research
03/26/2023

Lightweight Estimation of Hand Mesh and Biomechanically Feasible Kinematic Parameters

3D hand pose estimation is a long-standing challenge in both robotics an...
research
04/05/2018

Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images

We propose an end-to-end deep learning architecture that produces a 3D s...
research
03/28/2023

MeMaHand: Exploiting Mesh-Mano Interaction for Single Image Two-Hand Reconstruction

Existing methods proposed for hand reconstruction tasks usually paramete...
research
03/21/2020

Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data

We present a novel method for monocular hand shape and pose estimation a...

Please sign up or login with your details

Forgot password? Click here to reset