3D Hand Shape and Pose Estimation from a Single RGB Image

03/03/2019
by   Liuhao Ge, et al.
10

This work addresses a novel and challenging problem of estimating the full 3D hand shape and pose from a single RGB image. Most current methods in 3D hand analysis from monocular RGB images only focus on estimating the 3D locations of hand keypoints, which cannot fully express the 3D shape of hand. In contrast, we propose a Graph Convolutional Neural Network (Graph CNN) based method to reconstruct a full 3D mesh of hand surface that contains richer information of both 3D hand shape and pose. To train networks with full supervision, we create a large-scale synthetic dataset containing both ground truth 3D meshes and 3D poses. When fine-tuning the networks on real-world datasets without 3D ground truth, we propose a weakly-supervised approach by leveraging the depth map as a weak supervision in training. Through extensive evaluations on our proposed new datasets and two public datasets, we show that our proposed method can produce accurate and reasonable 3D hand mesh, and can achieve superior 3D hand pose estimation accuracy when compared with state-of-the-art methods.

READ FULL TEXT

page 1

page 4

page 8

page 12

research
04/03/2020

HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Map

3D hand shape and pose estimation from a single depth map is a new and c...
research
09/11/2019

Adaptive Wasserstein Hourglass for Weakly Supervised Hand Pose Estimation from Monocular RGB

Insufficient labeled training datasets is one of the bottlenecks of 3D h...
research
04/07/2017

Hand3D: Hand Pose Estimation using 3D Neural Network

We propose a novel 3D neural network architecture for 3D hand pose estim...
research
09/10/2019

FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Images

Estimating 3D hand pose from single RGB images is a highly ambiguous pro...
research
10/25/2022

THOR-Net: End-to-end Graformer-based Realistic Two Hands and Object Reconstruction with Self-supervision

Realistic reconstruction of two hands interacting with objects is a new ...
research
04/08/2019

Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Rendering

Estimating 3D hand meshes from single RGB images is challenging, due to ...
research
09/10/2018

Hand-tremor frequency estimation in videos

We focus on the problem of estimating human hand-tremor frequency from i...

Please sign up or login with your details

Forgot password? Click here to reset