3D Hand Pose Estimation using Simulation and Partial-Supervision with a Shared Latent Space

07/14/2018
by   Masoud Abdi, et al.
2

Tremendous amounts of expensive annotated data are a vital ingredient for state-of-the-art 3d hand pose estimation. Therefore, synthetic data has been popularized as annotations are automatically available. However, models trained only with synthetic samples do not generalize to real data, mainly due to the gap between the distribution of synthetic and real data. In this paper, we propose a novel method that seeks to predict the 3d position of the hand using both synthetic and partially-labeled real data. Accordingly, we form a shared latent space between three modalities: synthetic depth image, real depth image, and pose. We demonstrate that by carefully learning the shared latent space, we can find a regression model that is able to generalize to real data. As such, we show that our method produces accurate predictions in both semi-supervised and unsupervised settings. Additionally, the proposed model is capable of generating novel, meaningful, and consistent samples from all of the three domains. We evaluate our method qualitatively and quantitively on two highly competitive benchmarks (i.e., NYU and ICVL) and demonstrate its superiority over the state-of-the-art methods. The source code will be made available at https://github.com/masabdi/LSPS.

READ FULL TEXT

page 6

page 8

page 15

page 16

research
07/11/2019

Cross-Domain Complementary Learning with Synthetic Data for Multi-Person Part Segmentation

The success of supervised deep learning depends on the training labels. ...
research
03/30/2018

Cross-modal Deep Variational Hand Pose Estimation

The human hand moves in complex and high-dimensional ways, making estima...
research
09/19/2019

EPOSIT: An Absolute Pose Estimation Method for Pinhole and Fish-Eye Cameras

This paper presents a generic 6DOF camera pose estimation method, which ...
research
11/23/2018

MURAUER: Mapping Unlabeled Real Data for Label AUstERity

Data labeling for learning 3D hand pose estimation models is a huge effo...
research
07/04/2019

Sim2real transfer learning for 3D pose estimation: motion to the rescue

Simulation is an anonymous, low-bias source of data where annotation can...
research
06/07/2020

SharinGAN: Combining Synthetic and Real Data for Unsupervised Geometry Estimation

We propose a novel method for combining synthetic and real images when t...
research
10/08/2020

Deep SVBRDF Estimation on Real Materials

Recent work has demonstrated that deep learning approaches can successfu...

Please sign up or login with your details

Forgot password? Click here to reset