Personalizing Human Video Pose Estimation

11/20/2015
by   James Charles, et al.
0

We propose a personalized ConvNet pose estimator that automatically adapts itself to the uniqueness of a person's appearance to improve pose estimation in long videos. We make the following contributions: (i) we show that given a few high-precision pose annotations, e.g. from a generic ConvNet pose estimator, additional annotations can be generated throughout the video using a combination of image-based matching for temporally distant frames, and dense optical flow for temporally local frames; (ii) we develop an occlusion aware self-evaluation model that is able to automatically select the high-quality and reject the erroneous additional annotations; and (iii) we demonstrate that these high-quality annotations can be used to fine-tune a ConvNet pose estimator and thereby personalize it to lock on to key discriminative features of the person's appearance. The outcome is a substantial improvement in the pose estimates for the target video using the personalized ConvNet compared to the original generic ConvNet. Our method outperforms the state of the art (including top ConvNet methods) by a large margin on two standard benchmarks, as well as on a new challenging YouTube video dataset. Furthermore, we show that training from the automatically generated annotations can be used to improve the performance of a generic ConvNet on other benchmarks.

READ FULL TEXT

page 2

page 3

page 4

page 6

page 8

page 13

page 14

research
06/06/2019

Learning Temporal Pose Estimation from Sparsely-Labeled Videos

Modern approaches for multi-person pose estimation in video require larg...
research
06/09/2015

Flowing ConvNets for Human Pose Estimation in Videos

The objective of this work is human pose estimation in videos, where mul...
research
12/05/2019

15 Keypoints Is All You Need

Pose tracking is an important problem that requires identifying unique h...
research
01/12/2021

Temporally Guided Articulated Hand Pose Tracking in Surgical Videos

Articulated hand pose tracking is an underexplored problem that carries ...
research
11/30/2021

Semi-Supervised 3D Hand Shape and Pose Estimation with Label Propagation

To obtain 3D annotations, we are restricted to controlled environments o...
research
10/24/2019

TexturePose: Supervising Human Mesh Estimation with Texture Consistency

This work addresses the problem of model-based human pose estimation. Re...
research
10/10/2019

MetaPix: Few-Shot Video Retargeting

We address the task of unsupervised retargeting of human actions from on...

Please sign up or login with your details

Forgot password? Click here to reset