MonoCap: Monocular Human Motion Capture using a CNN Coupled with a Geometric Prior

01/09/2017
by   Xiaowei Zhou, et al.
0

Recovering 3D full-body human pose is a challenging problem with many applications. It has been successfully addressed by motion capture systems with body worn markers and multiple cameras. In this paper, we address the more challenging case of not only using a single camera but also not leveraging markers: going directly from 2D appearance to 3D geometry. Deep learning approaches have shown remarkable abilities to discriminatively learn 2D appearance features. The missing piece is how to integrate 2D, 3D and temporal information to recover 3D geometry and account for the uncertainties arising from the discriminative model. We introduce a novel approach that treats 2D joint locations as latent variables, whose uncertainty distributions are given by a deep fully convolutional network. The unknown 3D poses are modeled by a sparse representation and the 3D parameter estimates are realized via an Expectation-Maximization algorithm, where it is shown that the 2D joint location uncertainties can be conveniently marginalized out during inference. Extensive evaluation on benchmark datasets shows that the proposed approach achieves greater accuracy over state-of-the-art baselines. Notably, the proposed approach does not require synchronized 2D-3D data for training and is applicable to "in-the-wild" images, which is demonstrated with the MPII dataset.

READ FULL TEXT

page 2

page 5

page 9

page 10

page 11

page 12

research
11/30/2015

Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video

This paper addresses the challenge of 3D full-body human pose estimation...
research
12/04/2018

Monocular Total Capture: Posing Face, Body, and Hands in the Wild

We present the first method to capture the 3D total motion of a target p...
research
06/11/2014

Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation

This paper proposes a new hybrid architecture that consists of a deep Co...
research
12/18/2019

ActiveMoCap: Optimized Drone Flight for Active Human Motion Capture

The accuracy of monocular 3D human pose estimation depends on the viewpo...
research
07/21/2020

Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop

We introduce an automatic, end-to-end method for recovering the 3D pose ...
research
09/09/2023

Mirror-Aware Neural Humans

Human motion capture either requires multi-camera systems or is unreliab...
research
03/02/2022

H4D: Human 4D Modeling by Learning Neural Compositional Representation

Despite the impressive results achieved by deep learning based 3D recons...

Please sign up or login with your details

Forgot password? Click here to reset