DeepHPS: End-to-end Estimation of 3D Hand Pose and Shape by Learning from Synthetic Depth

by   Jameel Malik, et al.

Articulated hand pose and shape estimation is an important problem for vision-based applications such as augmented reality and animation. In contrast to the existing methods which optimize only for joint positions, we propose a fully supervised deep network which learns to jointly estimate a full 3D hand mesh representation and pose from a single depth image. To this end, a CNN architecture is employed to estimate parametric representations i.e. hand pose, bone scales and complex shape parameters. Then, a novel hand pose and shape layer, embedded inside our deep framework, produces 3D joint positions and hand mesh. Lack of sufficient training data with varying hand shapes limits the generalized performance of learning based methods. Also, manually annotating real data is suboptimal. Therefore, we present SynHand5M: a million-scale synthetic dataset with accurate joint annotations, segmentation masks and mesh files of depth maps. Among model based learning (hybrid) methods, we show improved results on our dataset and two of the public benchmarks i.e. NYU and ICVL. Also, by employing a joint training strategy with real and synthetic data, we recover 3D hand mesh and pose from real images in 3.7ms.


HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Map

3D hand shape and pose estimation from a single depth map is a new and c...

Simultaneous Hand Pose and Skeleton Bone-Lengths Estimation from a Single Depth Image

Articulated hand pose estimation is a challenging task for human-compute...

3D Hand Shape and Pose from Images in the Wild

We present in this work the first end-to-end deep learning based method ...

Lightweight Estimation of Hand Mesh and Biomechanically Feasible Kinematic Parameters

3D hand pose estimation is a long-standing challenge in both robotics an...

Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data

We present a novel method for monocular hand shape and pose estimation a...

Mask2Hand: Learning to Predict the 3D Hand Pose and Shape from Shadow

We present a self-trainable method, Mask2Hand, which learns to solve the...

Creatures great and SMAL: Recovering the shape and motion of animals from video

We present a system to recover the 3D shape and motion of a wide variety...

Please sign up or login with your details

Forgot password? Click here to reset