Space-Time-Separable Graph Convolutional Network for Pose Forecasting

10/09/2021
by   Theodoros Sofianos, et al.
0

Human pose forecasting is a complex structured-data sequence-modelling task, which has received increasing attention, also due to numerous potential applications. Research has mainly addressed the temporal dimension as time series and the interaction of human body joints with a kinematic tree or by a graph. This has decoupled the two aspects and leveraged progress from the relevant fields, but it has also limited the understanding of the complex structural joint spatio-temporal dynamics of the human pose. Here we propose a novel Space-Time-Separable Graph Convolutional Network (STS-GCN) for pose forecasting. For the first time, STS-GCN models the human pose dynamics only with a graph convolutional network (GCN), including the temporal evolution and the spatial joint interaction within a single-graph framework, which allows the cross-talk of motion and spatial correlations. Concurrently, STS-GCN is the first space-time-separable GCN: the space-time graph connectivity is factored into space and time affinity matrices, which bottlenecks the space-time cross-talk, while enabling full joint-joint and time-time correlations. Both affinity matrices are learnt end-to-end, which results in connections substantially deviating from the standard kinematic tree and the linear-time time series. In experimental evaluation on three complex, recent and large-scale benchmarks, Human3.6M [Ionescu et al. TPAMI'14], AMASS [Mahmood et al. ICCV'19] and 3DPW [Von Marcard et al. ECCV'18], STS-GCN outperforms the state-of-the-art, surpassing the current best technique [Mao et al. ECCV'20] by over 32 requiring 1.7 illustrate the graph interactions by the factored joint-joint and time-time learnt graph connections. Our source code is available at: https://github.com/FraLuca/STSGCN

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/20/2020

A3T-GCN: Attention Temporal Graph Convolutional Network for Traffic Forecasting

Accurate real-time traffic forecasting is a core technological problem a...
research
07/24/2022

Pose Forecasting in Industrial Human-Robot Collaboration

Pushing back the frontiers of collaborative robots in industrial environ...
research
04/12/2023

Best Practices for 2-Body Pose Forecasting

The task of collaborative human pose forecasting stands for predicting t...
research
08/16/2021

MSR-GCN: Multi-Scale Residual Graph Convolution Networks for Human Motion Prediction

Human motion prediction is a challenging task due to the stochasticity a...
research
09/15/2021

Learning Dynamical Human-Joint Affinity for 3D Pose Estimation in Videos

Graph Convolution Network (GCN) has been successfully used for 3D human ...
research
08/10/2023

Double-chain Constraints for 3D Human Pose Estimation in Images and Videos

Reconstructing 3D poses from 2D poses lacking depth information is parti...
research
02/17/2023

3D Human Pose Lifting with Grid Convolution

Existing lifting networks for regressing 3D human poses from 2D single-v...

Please sign up or login with your details

Forgot password? Click here to reset