Probabilistic Modeling for Human Mesh Recovery

by   Nikos Kolotouros, et al.

This paper focuses on the problem of 3D human reconstruction from 2D evidence. Although this is an inherently ambiguous problem, the majority of recent works avoid the uncertainty modeling and typically regress a single estimate for a given input. In contrast to that, in this work, we propose to embrace the reconstruction ambiguity and we recast the problem as learning a mapping from the input to a distribution of plausible 3D poses. Our approach is based on the normalizing flows model and offers a series of advantages. For conventional applications, where a single 3D estimate is required, our formulation allows for efficient mode computation. Using the mode leads to performance that is comparable with the state of the art among deterministic unimodal regression models. Simultaneously, since we have access to the likelihood of each sample, we demonstrate that our model is useful in a series of downstream tasks, where we leverage the probabilistic nature of the prediction as a tool for more accurate estimation. These tasks include reconstruction from multiple uncalibrated views, as well as human model fitting, where our model acts as a powerful image-based prior for mesh recovery. Our results validate the importance of probabilistic modeling, and indicate state-of-the-art performance across a variety of settings. Code and models are available at: nkolot/projects/prohmr.


page 2

page 7

page 8


Generative Approach for Probabilistic Human Mesh Recovery using Diffusion Models

This work focuses on the problem of reconstructing a 3D human body mesh ...

Distribution-Aligned Diffusion for Human Mesh Recovery

Recovering a 3D human mesh from a single RGB image is a challenging task...

I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Image

Most of the previous image-based 3D human pose and mesh estimation metho...

Leveraging MoCap Data for Human Mesh Recovery

Training state-of-the-art models for human body pose and shape recovery ...

POCO: 3D Pose and Shape Estimation with Confidence

The regression of 3D Human Pose and Shape (HPS) from an image is becomin...

Learning Local Recurrent Models for Human Mesh Recovery

We consider the problem of estimating frame-level full human body meshes...

3D Multi-bodies: Fitting Sets of Plausible 3D Human Models to Ambiguous Image Data

We consider the problem of obtaining dense 3D reconstructions of humans ...

Please sign up or login with your details

Forgot password? Click here to reset