3D Skeleton-based Human Motion Prediction with Manifold-Aware GAN

03/01/2022
by   Baptiste Chopin, et al.
0

In this work we propose a novel solution for 3D skeleton-based human motion prediction. The objective of this task consists in forecasting future human poses based on a prior skeleton pose sequence. This involves solving two main challenges still present in recent literature; (1) discontinuity of the predicted motion which results in unrealistic motions and (2) performance deterioration in long-term horizons resulting from error accumulation across time. We tackle these issues by using a compact manifold-valued representation of 3D human skeleton motion. Specifically, we model the temporal evolution of the 3D poses as trajectory, what allows us to map human motions to single points on a sphere manifold. Using such a compact representation avoids error accumulation and provides robust representation for long-term prediction while ensuring the smoothness and the coherence of the whole motion. To learn these non-Euclidean representations, we build a manifold-aware Wasserstein generative adversarial model that captures the temporal and spatial dependencies of human motion through different losses. Experiments have been conducted on CMU MoCap and Human 3.6M datasets and demonstrate the superiority of our approach over the state-of-the-art both in short and long term horizons. The smoothness of the generated motion is highlighted in the qualitative results.

READ FULL TEXT
research
05/18/2021

Human Motion Prediction Using Manifold-Aware Wasserstein GAN

Human motion prediction aims to forecast future human poses given a prio...
research
12/19/2020

GlocalNet: Class-aware Long-term Human Motion Synthesis

Synthesis of long-term human motion skeleton sequences is essential to a...
research
12/13/2018

Human Motion Prediction via Spatio-Temporal Inpainting

We propose a Generative Adversarial Network (GAN) to forecast 3D human m...
research
09/21/2021

Skeleton-Graph: Long-Term 3D Motion Prediction From 2D Observations Using Deep Spatio-Temporal Graph CNNs

Several applications such as autonomous driving, augmented reality and v...
research
08/20/2019

Spatio-temporal Manifold Learning for Human Motions via Long-horizon Modeling

Data-driven modeling of human motions is ubiquitous in computer graphics...
research
11/27/2017

HP-GAN: Probabilistic 3D human motion prediction via GAN

Predicting and understanding human motion dynamics has many applications...
research
12/06/2018

Unsupervised Feature Learning of Human Actions as Trajectories in Pose Embedding Manifold

An unsupervised human action modeling framework can provide useful pose-...

Please sign up or login with your details

Forgot password? Click here to reset