Skeletor: Skeletal Transformers for Robust Body-Pose Estimation

04/23/2021
by   Tao Jiang, et al.
5

Predicting 3D human pose from a single monoscopic video can be highly challenging due to factors such as low resolution, motion blur and occlusion, in addition to the fundamental ambiguity in estimating 3D from 2D. Approaches that directly regress the 3D pose from independent images can be particularly susceptible to these factors and result in jitter, noise and/or inconsistencies in skeletal estimation. Much of which can be overcome if the temporal evolution of the scene and skeleton are taken into account. However, rather than tracking body parts and trying to temporally smooth them, we propose a novel transformer based network that can learn a distribution over both pose and motion in an unsupervised fashion. We call our approach Skeletor. Skeletor overcomes inaccuracies in detection and corrects partial or entire skeleton corruption. Skeletor uses strong priors learn from on 25 million frames to correct skeleton sequences smoothly and consistently. Skeletor can achieve this as it implicitly learns the spatio-temporal context of human motion via a transformer based neural network. Extensive experiments show that Skeletor achieves improved performance on 3D human pose estimation and further provides benefits for downstream tasks such as sign language translation.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/25/2019

Out of the Box: A combined approach for handling occlusion in Human Pose Estimation

Human Pose estimation is a challenging problem, especially in the case o...
research
10/12/2022

MotionBERT: Unified Pretraining for Human Motion Analysis

We present MotionBERT, a unified pretraining framework, to tackle differ...
research
08/18/2023

Unsupervised 3D Pose Estimation with Non-Rigid Structure-from-Motion Modeling

Most of the previous 3D human pose estimation work relied on the powerfu...
research
07/02/2020

JUMPS: Joints Upsampling Method for Pose Sequences

Human Pose Estimation is a low-level task useful for surveillance, human...
research
03/28/2022

Semantic Motion Correction Via Iterative Nonlinear Optimization and Animation

Here, we present an end-to-end method to create 2D animation for a goalk...
research
06/09/2022

Building Spatio-temporal Transformers for Egocentric 3D Pose Estimation

Egocentric 3D human pose estimation (HPE) from images is challenging due...
research
10/15/2021

3D Human Pose Estimation for Free-form Activity Using WiFi Signals

WiFi human sensing has become increasingly attractive in enabling emergi...

Please sign up or login with your details

Forgot password? Click here to reset