Spatio-Temporal Branching for Motion Prediction using Motion Increments

08/02/2023
by   Jiexin Wang, et al.
0

Human motion prediction (HMP) has emerged as a popular research topic due to its diverse applications, but it remains a challenging task due to the stochastic and aperiodic nature of future poses. Traditional methods rely on hand-crafted features and machine learning techniques, which often struggle to model the complex dynamics of human motion. Recent deep learning-based methods have achieved success by learning spatio-temporal representations of motion, but these models often overlook the reliability of motion data. Additionally, the temporal and spatial dependencies of skeleton nodes are distinct. The temporal relationship captures motion information over time, while the spatial relationship describes body structure and the relationships between different nodes. In this paper, we propose a novel spatio-temporal branching network using incremental information for HMP, which decouples the learning of temporal-domain and spatial-domain features, extracts more motion information, and achieves complementary cross-domain knowledge learning through knowledge distillation. Our approach effectively reduces noise interference and provides more expressive information for characterizing motion by separately extracting temporal and spatial features. We evaluate our approach on standard HMP benchmarks and outperform state-of-the-art methods in terms of prediction accuracy.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/13/2013

Learning to encode motion using spatio-temporal synchrony

We consider the task of learning to extract motion from videos. To this ...
research
11/06/2019

Predicting Long-Term Skeletal Motions by a Spatio-Temporal Hierarchical Recurrent Network

The primary goal of skeletal motion prediction is to generate future mot...
research
03/11/2023

SPOTR: Spatio-temporal Pose Transformers for Human Motion Prediction

3D human motion prediction is a research area of high significance and a...
research
08/25/2021

Multiscale Spatio-Temporal Graph Neural Networks for 3D Skeleton-Based Motion Prediction

We propose a multiscale spatio-temporal graph neural network (MST-GNN) t...
research
06/24/2019

LMVP: Video Predictor with Leaked Motion Information

We propose a Leaked Motion Video Predictor (LMVP) to predict future fram...
research
04/20/2022

A Probabilistic Time-Evolving Approach to Scanpath Prediction

Human visual attention is a complex phenomenon that has been studied for...
research
05/25/2023

Text-to-Motion Retrieval: Towards Joint Understanding of Human Motion Data and Natural Language

Due to recent advances in pose-estimation methods, human motion can be e...

Please sign up or login with your details

Forgot password? Click here to reset