General Automatic Human Shape and Motion Capture Using Volumetric Contour Cues

by   Helge Rhodin, et al.

Markerless motion capture algorithms require a 3D body with properly personalized skeleton dimension and/or body shape and appearance to successfully track a person. Unfortunately, many tracking methods consider model personalization a different problem and use manual or semi-automatic model initialization, which greatly reduces applicability. In this paper, we propose a fully automatic algorithm that jointly creates a rigged actor model commonly used for animation - skeleton, volumetric shape, appearance, and optionally a body surface - and estimates the actor's motion from multi-view video input only. The approach is rigorously designed to work on footage of general outdoor scenes recorded with very few cameras and without background subtraction. Our method uses a new image formation model with analytic visibility and analytically differentiable alignment energy. For reconstruction, 3D body shape is approximated as Gaussian density field. For pose and shape estimation, we minimize a new edge-based alignment energy inspired by volume raycasting in an absorbing medium. We further propose a new statistical human body model that represents the body surface, volumetric Gaussian density, as well as variability in skeleton shape. Given any multi-view sequence, our method jointly optimizes the pose and shape parameters of this model fully automatically in a spatiotemporal way.


page 2

page 6

page 9

page 10

page 12

page 13

page 14


Towards Accurate Markerless Human Shape and Pose Estimation over Time

Existing marker-less motion capture methods often assume known backgroun...

Model-based Outdoor Performance Capture

We propose a new model-based method to accurately reconstruct human perf...

A-NeRF: Surface-free Human 3D Pose Refinement via Neural Rendering

While deep learning has reshaped the classical motion capture pipeline, ...

Robust Temporally Coherent Laplacian Protrusion Segmentation of 3D Articulated Bodies

In motion analysis and understanding it is important to be able to fit a...

Shape Animation with Combined Captured and Simulated Dynamics

We present a novel volumetric animation generation framework to create n...

EgoCap: Egocentric Marker-less Motion Capture with Two Fisheye Cameras (Extended Abstract)

Marker-based and marker-less optical skeletal motion-capture methods use...

TAVA: Template-free Animatable Volumetric Actors

Coordinate-based volumetric representations have the potential to genera...

Please sign up or login with your details

Forgot password? Click here to reset