Learning Spatial Context with Graph Neural Network for Multi-Person Pose Grouping

by   Jiahao Lin, et al.

Bottom-up approaches for image-based multi-person pose estimation consist of two stages: (1) keypoint detection and (2) grouping of the detected keypoints to form person instances. Current grouping approaches rely on learned embedding from only visual features that completely ignore the spatial configuration of human poses. In this work, we formulate the grouping task as a graph partitioning problem, where we learn the affinity matrix with a Graph Neural Network (GNN). More specifically, we design a Geometry-aware Association GNN that utilizes spatial information of the keypoints and learns local affinity from the global context. The learned geometry-based affinity is further fused with appearance-based affinity to achieve robust keypoint association. Spectral clustering is used to partition the graph for the formation of the pose instances. Experimental results on two benchmark datasets show that our proposed method outperforms existing appearance-only grouping frameworks, which shows the effectiveness of utilizing spatial context for robust grouping. Source code is available at: https://github.com/jiahaoLjh/PoseGrouping.



There are no comments yet.


page 1

page 3

page 5

page 6


Differentiable Hierarchical Graph Grouping for Multi-Person Pose Estimation

Multi-person pose estimation is challenging because it localizes body ke...

Multi-person Articulated Tracking with Spatial and Temporal Embeddings

We propose a unified framework for multi-person pose estimation and trac...

Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression

In this paper, we are interested in the bottom-up paradigm of estimating...

The Center of Attention: Center-Keypoint Grouping via Attention for Multi-Person Pose Estimation

We introduce CenterGroup, an attention-based framework to estimate human...

Greedy Offset-Guided Keypoint Grouping for Human Pose Estimation

We propose a simple yet reliable bottom-up approach with a good trade-of...

Learning Skeletal Graph Neural Networks for Hard 3D Pose Estimation

Various deep learning techniques have been proposed to solve the single-...

A Global to Local Double Embedding Method for Multi-person Pose Estimation

Multi-person pose estimation is a fundamental and challenging problem to...

Code Repositories


Code for "Learning Spatial Context with Graph Neural Network for Multi-Person Pose Grouping"

view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.