DeepAI AI Chat
Log In Sign Up

Self-Supervised 3D Hand Pose Estimation from monocular RGB via Contrastive Learning

by   Adrian Spurr, et al.

Acquiring accurate 3D annotated data for hand pose estimation is a notoriously difficult problem. This typically requires complex multi-camera setups and controlled conditions, which in turn creates a domain gap that is hard to bridge to fully unconstrained settings. Encouraged by the success of contrastive learning on image classification tasks, we propose a new self-supervised method for the structured regression task of 3D hand pose estimation. Contrastive learning makes use of unlabeled data for the purpose of representation learning via a loss formulation that encourages the learned feature representations to be invariant under any image transformation. For 3D hand pose estimation, it too is desirable to have invariance to appearance transformation such as color jitter. However, the task requires equivariance under affine transformations, such as rotation and translation. To address this issue, we propose an equivariant contrastive objective and demonstrate its effectiveness in the context of 3D hand pose estimation. We experimentally investigate the impact of invariant and equivariant contrastive objectives and show that learning equivariant features leads to better representations for the task of 3D hand pose estimation. Furthermore, we show that a standard ResNet-152, trained on additional unlabeled data, attains an improvement of 7.6% in PA-EPE on FreiHAND and thus achieves state-of-the-art performance without any task specific, specialized architectures.


page 1

page 6


Image-free Domain Generalization via CLIP for 3D Hand Pose Estimation

RGB-based 3D hand pose estimation has been successful for decades thanks...

TempCLR: Reconstructing Hands via Time-Coherent Contrastive Learning

We introduce TempCLR, a new time-coherent contrastive learning approach ...

Self-Supervised Multi-View Synchronization Learning for 3D Pose Estimation

Current state-of-the-art methods cast monocular 3D human pose estimation...

Explicit homography estimation improves contrastive self-supervised learning

The typical contrastive self-supervised algorithm uses a similarity meas...

Rotation-invariant Mixed Graphical Model Network for 2D Hand Pose Estimation

In this paper, we propose a new architecture named Rotation-invariant Mi...

ScatSimCLR: self-supervised contrastive learning with pretext task regularization for small-scale datasets

In this paper, we consider a problem of self-supervised learning for sma...

A Contrastive Objective for Learning Disentangled Representations

Learning representations of images that are invariant to sensitive or un...