PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Model

03/22/2018
by   George Papandreou, et al.
1

We present a box-free bottom-up approach for the tasks of pose estimation and instance segmentation of people in multi-person images using an efficient single-shot model. The proposed PersonLab model tackles both semantic-level reasoning and object-part associations using part-based modeling. Our model employs a convolutional network which learns to detect individual keypoints and predict their relative displacements, allowing us to group keypoints into person pose instances. Further, we propose a part-induced geometric embedding descriptor which allows us to associate semantic person pixels with their corresponding person instance, delivering instance-level person segmentations. Our system is based on a fully-convolutional architecture and allows for efficient inference, with runtime essentially independent of the number of people present in the scene. Trained on COCO data alone, our system achieves COCO test-dev keypoint average precision of 0.665 using single-scale inference and 0.687 using multi-scale inference, significantly outperforming all previous bottom-up pose estimation systems. We are also the first bottom-up method to report competitive results for the person class in the COCO instance segmentation task, achieving a person category average precision of 0.417.

READ FULL TEXT

page 5

page 6

page 8

page 9

page 15

07/20/2018

Bounding Box Embedding for Single Shot Person Instance Segmentation

We present a bottom-up approach for the task of object instance segmenta...
11/16/2016

Associative Embedding: End-to-End Learning for Joint Detection and Grouping

We introduce associative embedding, a novel method for supervising convo...
04/04/2017

Pose2Instance: Harnessing Keypoints for Person Instance Segmentation

Human keypoints are a well-studied representation of people.We explore h...
01/06/2017

Towards Accurate Multi-person Pose Estimation in the Wild

We propose a method for multi-person detection and 2-D pose estimation t...
08/15/2022

Uni6Dv2: Noise Elimination for 6D Pose Estimation

Few prior 6D pose estimation methods use a backbone network to extract f...
03/30/2017

Semantic Instance Segmentation via Deep Metric Learning

We propose a new method for semantic instance segmentation, by first com...
11/24/2016

Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields

We present an approach to efficiently detect the 2D pose of multiple peo...

Code Repositories

KerasPersonLab

Keras-tensorflow implementation of PersonLab (https://arxiv.org/abs/1803.08225)


view repo

Please sign up or login with your details

Forgot password? Click here to reset