Flowing ConvNets for Human Pose Estimation in Videos

06/09/2015
by   Tomas Pfister, et al.
0

The objective of this work is human pose estimation in videos, where multiple frames are available. We investigate a ConvNet architecture that is able to benefit from temporal context by combining information across the multiple frames using optical flow. To this end we propose a network architecture with the following novelties: (i) a deeper network than previously investigated for regressing heatmaps; (ii) spatial fusion layers that learn an implicit spatial model; (iii) optical flow is used to align heatmap predictions from neighbouring frames; and (iv) a final parametric pooling layer which learns to combine the aligned heatmaps into a pooled confidence map. We show that this architecture outperforms a number of others, including one that uses optical flow solely at the input layers, one that regresses joint coordinates directly, and one that predicts heatmaps without spatial fusion. The new architecture outperforms the state of the art by a large margin on three video pose estimation datasets, including the very challenging Poses in the Wild dataset, and outperforms other deep methods that don't use a graphical model on the single-image FLIC benchmark (and also Chen & Yuille and Tompson et al. in the high precision region).

READ FULL TEXT

page 2

page 3

page 4

page 5

page 9

page 10

page 11

page 12

research
10/27/2022

Bootstrapping Human Optical Flow and Pose

We propose a bootstrapping framework to enhance human optical flow and p...
research
10/22/2021

Deep Two-Stream Video Inference for Human Body Pose and Shape Estimation

Several video-based 3D pose and shape estimation algorithms have been pr...
research
11/20/2015

Personalizing Human Video Pose Estimation

We propose a personalized ConvNet pose estimator that automatically adap...
research
01/29/2019

Visual Rhythm Prediction with Feature-Aligning Network

In this paper, we propose a data-driven visual rhythm prediction method,...
research
05/28/2020

3D human pose estimation with adaptive receptive fields and dilated temporal convolutions

In this work, we demonstrate that receptive fields in 3D pose estimation...
research
06/23/2023

Shape-Constraint Recurrent Flow for 6D Object Pose Estimation

Most recent 6D object pose methods use 2D optical flow to refine their r...
research
05/23/2019

Pose estimator and tracker using temporal flow maps for limbs

For human pose estimation in videos, it is significant how to use tempor...

Please sign up or login with your details

Forgot password? Click here to reset