DeepAI AI Chat
Log In Sign Up

Weakly Supervised 3D Hand Pose Estimation via Biomechanical Constraints

by   Adrian Spurr, et al.
ETH Zurich

Estimating 3D hand pose from 2D images is a difficult, inverse problem due to the inherent scale and depth ambiguities. Current state-of-the-art methods train fully supervised deep neural networks with 3D ground-truth data. However, acquiring 3D annotations is expensive, typically requiring calibrated multi-view setups or labor intensive manual annotations. While annotations of 2D keypoints are much easier to obtain, how to efficiently leverage such weakly-supervised data to improve the task of 3D hand pose prediction remains an important open question. The key difficulty stems from the fact that direct application of additional 2D supervision mostly benefits the 2D proxy objective but does little to alleviate the depth and scale ambiguities. Embracing this challenge we propose a set of novel losses. We show by extensive experiments that our proposed constraints significantly reduce the depth ambiguity and allow the network to more effectively leverage additional 2D annotated images. For example, on the challenging freiHAND dataset using additional 2D annotation without our proposed biomechanical constraints reduces the depth error by only 15%, whereas the error is reduced significantly by 50% when the proposed biomechanical constraints are used.


page 1

page 2

page 3

page 4


Distill Knowledge from NRSfM for Weakly Supervised 3D Pose Learning

We propose to learn a 3D pose estimator by distilling knowledge from Non...

Lifting 2d Human Pose to 3d : A Weakly Supervised Approach

Estimating 3d human pose from monocular images is a challenging problem ...

Adaptive Wasserstein Hourglass for Weakly Supervised Hand Pose Estimation from Monocular RGB

Insufficient labeled training datasets is one of the bottlenecks of 3D h...

Hand Pose Estimation through Semi-Supervised and Weakly-Supervised Learning

We propose a method for hand pose estimation based on a deep regressor t...

Model-based 3D Hand Reconstruction via Self-Supervised Learning

Reconstructing a 3D hand from a single-view RGB image is challenging due...

End-to-end Weakly-supervised Multiple 3D Hand Mesh Reconstruction from Single Image

In this paper, we consider the challenging task of simultaneously locati...