Task-Generic Hierarchical Human Motion Prior using VAEs

06/07/2021
by   Jiaman Li, et al.
15

A deep generative model that describes human motions can benefit a wide range of fundamental computer vision and graphics tasks, such as providing robustness to video-based human pose estimation, predicting complete body movements for motion capture systems during occlusions, and assisting key frame animation with plausible movements. In this paper, we present a method for learning complex human motions independent of specific tasks using a combined global and local latent space to facilitate coarse and fine-grained modeling. Specifically, we propose a hierarchical motion variational autoencoder (HM-VAE) that consists of a 2-level hierarchical latent space. While the global latent space captures the overall global body motion, the local latent space enables to capture the refined poses of the different body parts. We demonstrate the effectiveness of our hierarchical motion variational autoencoder in a variety of tasks including video-based human pose estimation, motion completion from partial observations, and motion synthesis from sparse key-frames. Even though, our model has not been trained for any of these tasks specifically, it provides superior performance than task-specific alternatives. Our general-purpose human motion prior model can fix corrupted human body animations and generate complete movements from incomplete observations.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

05/10/2021

HuMoR: 3D Human Motion Model for Robust Pose Estimation

We introduce HuMoR: a 3D Human Motion Model for Robust Estimation of tem...
08/09/2020

3D Human Motion Estimation via Motion Compression and Refinement

We develop a technique for generating smooth and accurate 3D human pose ...
05/21/2021

Driving-Signal Aware Full-Body Avatars

We present a learning-based method for building driving-signal aware ful...
03/18/2021

Future Frame Prediction for Robot-assisted Surgery

Predicting future frames for robotic surgical video is an interesting, i...
06/07/2021

Multi-frame sequence generator of 4D human body motion

We examine the problem of generating temporally and spatially dense 4D h...
12/01/2020

We are More than Our Joints: Predicting how 3D Bodies Move

A key step towards understanding human behavior is the prediction of 3D ...
10/20/2020

Probabilistic Character Motion Synthesis using a Hierarchical Deep Latent Variable Model

We present a probabilistic framework to generate character animations ba...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.