HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Map

by   Jameel Malik, et al.

3D hand shape and pose estimation from a single depth map is a new and challenging computer vision problem with many applications. The state-of-the-art methods directly regress 3D hand meshes from 2D depth images via 2D convolutional neural networks, which leads to artefacts in the estimations due to perspective distortions in the images. In contrast, we propose a novel architecture with 3D convolutions trained in a weakly-supervised manner. The input to our method is a 3D voxelized depth map, and we rely on two hand shape representations. The first one is the 3D voxelized grid of the shape which is accurate but does not preserve the mesh topology and the number of mesh vertices. The second representation is the 3D hand surface which is less accurate but does not suffer from the limitations of the first representation. We combine the advantages of these two representations by registering the hand surface to the voxelized hand shape. In the extensive experiments, the proposed approach improves over the state of the art by 47.8 voxelized depth maps further enhances the accuracy of 3D hand pose estimation on real data. Our method produces visually more reasonable and realistic hand shapes on NYU and BigHand2.2M datasets compared to the existing approaches.


page 7

page 8


HandVoxNet++: 3D Hand Shape and Pose Estimation using Voxel-Based Neural Networks

3D hand shape and pose estimation from a single depth map is a new and c...

3D Hand Shape and Pose Estimation from a Single RGB Image

This work addresses a novel and challenging problem of estimating the fu...

DeepHPS: End-to-end Estimation of 3D Hand Pose and Shape by Learning from Synthetic Depth

Articulated hand pose and shape estimation is an important problem for v...

Lightweight Estimation of Hand Mesh and Biomechanically Feasible Kinematic Parameters

3D hand pose estimation is a long-standing challenge in both robotics an...

Hand3D: Hand Pose Estimation using 3D Neural Network

We propose a novel 3D neural network architecture for 3D hand pose estim...

Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild

We introduce a simple and effective network architecture for monocular 3...

Moulding Humans: Non-parametric 3D Human Shape Estimation from Single Images

In this paper, we tackle the problem of 3D human shape estimation from s...

Please sign up or login with your details

Forgot password? Click here to reset