Efficient Online Multi-Person 2D Pose Tracking with Recurrent Spatio-Temporal Affinity Fields

11/29/2018
by   Yaadhav Raaj, et al.
6

We present an online approach to efficiently and simultaneously detect and track the 2D pose of multiple people in a video sequence. We build upon Part Affinity Field (PAF) representation designed for static images, and propose an architecture that can encode and predict Spatio-Temporal Affinity Fields (STAF) across a video sequence. In particular, we propose a novel temporal topology cross-linked across limbs which can consistently handle body motions of a wide range of magnitudes. Additionally, we make the overall approach recurrent in nature, where the network ingests STAF heatmaps from previous frames and estimates those for the current frame. Our approach uses only online inference and tracking, and is currently the fastest and the most accurate bottom-up approach that is runtime invariant to the number of people in the scene and accuracy invariant to input frame rate of camera. Running at ∼30 fps on a single GPU at single scale, it achieves highly competitive results on the PoseTrack benchmarks.

READ FULL TEXT

page 1

page 3

page 5

page 6

page 8

research
11/24/2016

Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields

We present an approach to efficiently detect the 2D pose of multiple peo...
research
07/31/2016

A Data-driven Approach for Human Pose Tracking Based on Spatio-temporal Pictorial Structure

In this paper, we present a data-driven approach for human pose tracking...
research
03/03/2021

OpenPifPaf: Composite Fields for Semantic Keypoint Detection and Spatio-Temporal Association

Many image-based perception tasks can be formulated as detecting, associ...
research
12/18/2018

OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields

Realtime multi-person 2D pose estimation is a key component in enabling ...
research
12/05/2016

ArtTrack: Articulated Multi-person Tracking in the Wild

In this paper we propose an approach for articulated tracking of multipl...
research
04/30/2015

Predicting People's 3D Poses from Short Sequences

We propose an efficient approach to exploiting motion information from c...
research
03/22/2018

Learning to Detect and Track Visible and Occluded Body Joints in a Virtual World

Multi-People Tracking in an open-world setting requires a special effort...

Please sign up or login with your details

Forgot password? Click here to reset