Explicit Occlusion Reasoning for Multi-person 3D Human Pose Estimation

07/29/2022
by   Qihao Liu, et al.
0

Occlusion poses a great threat to monocular multi-person 3D human pose estimation due to large variability in terms of the shape, appearance, and position of occluders. While existing methods try to handle occlusion with pose priors/constraints, data augmentation, or implicit reasoning, they still fail to generalize to unseen poses or occlusion cases and may make large mistakes when multiple people are present. Inspired by the remarkable ability of humans to infer occluded joints from visible cues, we develop a method to explicitly model this process that significantly improves bottom-up multi-person human pose estimation with or without occlusions. First, we split the task into two subtasks: visible keypoints detection and occluded keypoints reasoning, and propose a Deeply Supervised Encoder Distillation (DSED) network to solve the second one. To train our model, we propose a Skeleton-guided human Shape Fitting (SSF) approach to generate pseudo occlusion labels on the existing datasets, enabling explicit occlusion reasoning. Experiments show that explicitly learning from occlusions improves human pose estimation. In addition, exploiting feature-level information of visible joints allows us to reason about occluded joints more accurately. Our method outperforms both the state-of-the-art top-down and bottom-up methods on several benchmarks.

READ FULL TEXT

page 2

page 13

research
08/28/2018

How Robust is 3D Human Pose Estimation to Occlusion?

Occlusion is commonplace in realistic human-robot shared environments, y...
research
11/29/2017

Occlusion-aware Hand Pose Estimation Using Hierarchical Mixture Density Network

Hand pose estimation is to predict the pose parameters representing a 3D...
research
06/06/2022

Mesh-based Dynamics with Occlusion Reasoning for Cloth Manipulation

Self-occlusion is challenging for cloth manipulation, as it makes it dif...
research
08/01/2021

LASOR: Learning Accurate 3D Human Pose and Shape Via Synthetic Occlusion-Aware Data and Neural Mesh Rendering

A key challenge in the task of human pose and shape estimation is occlus...
research
10/31/2020

Exploring Severe Occlusion: Multi-Person 3D Pose Estimation with Gated Convolution

3D human pose estimation (HPE) is crucial in many fields, such as human ...
research
03/21/2023

BoPR: Body-aware Part Regressor for Human Shape and Pose Estimation

This paper presents a novel approach for estimating human body shape and...
research
09/13/2023

LInKs "Lifting Independent Keypoints" – Partial Pose Lifting for Occlusion Handling with Improved Accuracy in 2D-3D Human Pose Estimation

We present LInKs, a novel unsupervised learning method to recover 3D hum...

Please sign up or login with your details

Forgot password? Click here to reset