Spatio-temporal motion completion using a sequence of latent primitives

06/27/2022
by   Mathieu Marsot, et al.
0

We propose a markerless performance capture method that computes a temporally coherent 4D representation of an actor deforming over time from a sparsely sampled sequence of untracked 3D point clouds. Our method proceeds by latent optimization with a spatio-temporal motion prior. Recently, task generic motion priors have been introduced and propose a coherent representation of human motion based on a single latent code, with encouraging results with short sequences and given temporal correspondences. Extending these methods to longer sequences without correspondences is all but straightforward. One latent code proves inefficient to encode longer term variability, and latent space optimization will be very susceptible to erroneous local minima due to possible inverted pose fittings. We address both problems by learning a motion prior that encodes a 4D human motion sequence into a sequence of latent primitives instead of one latent code. We also propose an additional mapping encoder which directly projects a sequence of point clouds into the learned latent space to provide a good initialization of the latent representation at inference time. Our temporal decoding from latent space is implicit and continuous in time, providing flexibility with temporal resolution. We show experimentally that our method outperforms state-of-the-art motion priors.

READ FULL TEXT
research
06/07/2021

Multi-frame sequence generator of 4D human body motion

We examine the problem of generating temporally and spatially dense 4D h...
research
04/18/2019

Human Motion Prediction via Pattern Completion in Latent Representation Space

Inspired by ideas in cognitive science, we propose a novel and general a...
research
07/16/2023

FourierHandFlow: Neural 4D Hand Representation Using Fourier Query Flow

Recent 4D shape representations model continuous temporal evolution of i...
research
03/29/2022

Learning a Structured Latent Space for Unsupervised Point Cloud Completion

Unsupervised point cloud completion aims at estimating the corresponding...
research
10/13/2019

Bayesian Neural Decoding Using A Diversity-Encouraging Latent Representation Learning Method

It is well established that temporal organization is critical to memory,...
research
07/03/2019

Tranquil Clouds: Neural Networks for Learning Temporally Coherent Features in Point Clouds

Point clouds, as a form of Lagrangian representation, allow for powerful...
research
04/14/2021

Temporally-Coherent Surface Reconstruction via Metric-Consistent Atlases

We propose a method for the unsupervised reconstruction of a temporally-...

Please sign up or login with your details

Forgot password? Click here to reset