Trajectory Space Factorization for Deep Video-Based 3D Human Pose Estimation

by   Jiahao Lin, et al.

Existing deep learning approaches on 3d human pose estimation for videos are either based on Recurrent or Convolutional Neural Networks (RNNs or CNNs). However, RNN-based frameworks can only tackle sequences with limited frames because sequential models are sensitive to bad frames and tend to drift over long sequences. Although existing CNN-based temporal frameworks attempt to address the sensitivity and drift problems by concurrently processing all input frames in the sequence, the existing state-of-the-art CNN-based framework is limited to 3d pose estimation of a single frame from a sequential input. In this paper, we propose a deep learning-based framework that utilizes matrix factorization for sequential 3d human poses estimation. Our approach processes all input frames concurrently to avoid the sensitivity and drift problems, and yet outputs the 3d pose estimates for every frame in the input sequence. More specifically, the 3d poses in all frames are represented as a motion matrix factorized into a trajectory bases matrix and a trajectory coefficient matrix. The trajectory bases matrix is precomputed from matrix factorization approaches such as Singular Value Decomposition (SVD) or Discrete Cosine Transform (DCT), and the problem of sequential 3d pose estimation is reduced to training a deep network to regress the trajectory coefficient matrix. We demonstrate the effectiveness of our framework on long sequences by achieving state-of-the-art performances on multiple benchmark datasets. Our source code is available at:


page 1

page 6

page 10


Video based Object 6D Pose Estimation using Transformers

We introduce a Transformer based 6D Object Pose Estimation framework Vid...

Learning to Estimate 3D Human Pose from Point Cloud

3D pose estimation is a challenging problem in computer vision. Most of ...

Temporal Feature Alignment and Mutual Information Maximization for Video-Based Human Pose Estimation

Multi-frame human pose estimation has long been a compelling and fundame...

Key Frame Proposal Network for Efficient Pose Estimation in Videos

Human pose estimation in video relies on local information by either est...

Frequency, Acceptability, and Selection: A case study of clause-embedding

We investigate the relationship between the frequency with which verbs a...

LSTM Pose Machines

We observed that recent state-of-the-art results on single image human p...

Hierarchical Attention and Graph Neural Networks: Toward Drift-Free Pose Estimation

The most commonly used method for addressing 3D geometric registration i...

Please sign up or login with your details

Forgot password? Click here to reset