View-Invariant, Occlusion-Robust Probabilistic Embedding for Human Pose

by   Ting Liu, et al.

Recognition of human poses and activities is crucial for autonomous systems to interact smoothly with people. However, cameras generally capture human poses in 2D as images and videos, which can have significant appearance variations across viewpoints. To address this, we explore recognizing similarity in 3D human body poses from 2D information, which has not been well-studied in existing works. Here, we propose an approach to learning a compact view-invariant embedding space from 2D body joint keypoints, without explicitly predicting 3D poses. Input ambiguities of 2D poses from projection and occlusion are difficult to represent through a deterministic mapping, and therefore we use probabilistic embeddings. In order to enable our embeddings to work with partially visible input keypoints, we further investigate different keypoint occlusion augmentation strategies during training. Experimental results show that our embedding model achieves higher accuracy when retrieving similar poses across different camera views, in comparison with 3D pose estimation models. We further show that keypoint occlusion augmentation during training significantly improves retrieval performance on partial 2D input poses. Results on action recognition and video alignment demonstrate that our embeddings, without any additional training, achieves competitive performance relative to other models specifically trained for each task.


page 11

page 13

page 15


View-Invariant Probabilistic Embedding for Human Pose

Depictions of similar human body configurations can vary with changing v...

Mirror-Aware Neural Humans

Human motion capture either requires multi-camera systems or is unreliab...

View Invariant 3D Human Pose Estimation

The recent success of deep networks has significantly advanced 3D human ...

Learning from Abstract Images: on the Importance of Occlusion in a Minimalist Encoding of Human Poses

Existing 2D-to-3D pose lifting networks suffer from poor performance in ...

Learning Human Pose Models from Synthesized Data for Robust RGB-D Action Recognition

We propose Human Pose Models that represent RGB and depth images of huma...

Automatic Analysis of Human Body Representations in Western Art

The way the human body is depicted in classical and modern paintings is ...

How to Train Your Robust Human Pose Estimator: Pay Attention to the Constraint Cue

Both appearance cue and constraint cue are important in human pose estim...

Please sign up or login with your details

Forgot password? Click here to reset