DeepAI AI Chat
Log In Sign Up

Adversarial Motion Modelling helps Semi-supervised Hand Pose Estimation

06/10/2021
by   Adrian Spurr, et al.
0

Hand pose estimation is difficult due to different environmental conditions, object- and self-occlusion as well as diversity in hand shape and appearance. Exhaustively covering this wide range of factors in fully annotated datasets has remained impractical, posing significant challenges for generalization of supervised methods. Embracing this challenge, we propose to combine ideas from adversarial training and motion modelling to tap into unlabeled videos. To this end we propose what to the best of our knowledge is the first motion model for hands and show that an adversarial formulation leads to better generalization properties of the hand pose estimator via semi-supervised training on unlabeled video sequences. In this setting, the pose predictor must produce a valid sequence of hand poses, as determined by a discriminative adversary. This adversary reasons both on the structural as well as temporal domain, effectively exploiting the spatio-temporal structure in the task. The main advantage of our approach is that we can make use of unpaired videos and joint sequence data both of which are much easier to attain than paired training data. We perform extensive evaluation, investigating essential components needed for the proposed framework and empirically demonstrate in two challenging settings that the proposed approach leads to significant improvements in pose estimation accuracy. In the lowest label setting, we attain an improvement of 40% in absolute mean joint error.

READ FULL TEXT

page 1

page 4

page 6

11/30/2021

Semi-Supervised 3D Hand Shape and Pose Estimation with Label Propagation

To obtain 3D annotations, we are restricted to controlled environments o...
11/28/2018

3D human pose estimation in video with temporal convolutions and semi-supervised training

In this work, we demonstrate that 3D poses in video can be effectively e...
07/16/2021

Semi-supervised 3D Hand-Object Pose Estimation via Pose Dictionary Learning

3D hand-object pose estimation is an important issue to understand the i...
07/20/2022

OTPose: Occlusion-Aware Transformer for Pose Estimation in Sparsely-Labeled Videos

Although many approaches for multi-human pose estimation in videos have ...
11/23/2017

Exploiting temporal information for 3D pose estimation

In this work, we address the problem of 3D human pose estimation from a ...
12/21/2020

Unsupervised Domain Adaptation with Temporal-Consistent Self-Training for 3D Hand-Object Joint Reconstruction

Deep learning-solutions for hand-object 3D pose and shape estimation are...
03/21/2020

Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data

We present a novel method for monocular hand shape and pose estimation a...