DeepAI AI Chat
Log In Sign Up

Neural Descent for Visual 3D Human Pose and Shape

by   Andrei Zanfir, et al.

We present deep neural network methodology to reconstruct the 3d pose and shape of people, given an input RGB image. We rely on a recently introduced, expressivefull body statistical 3d human model, GHUM, trained end-to-end, and learn to reconstruct its pose and shape state in a self-supervised regime. Central to our methodology, is a learning to learn and optimize approach, referred to as HUmanNeural Descent (HUND), which avoids both second-order differentiation when training the model parameters,and expensive state gradient descent in order to accurately minimize a semantic differentiable rendering loss at test time. Instead, we rely on novel recurrent stages to update the pose and shape parameters such that not only losses are minimized effectively, but the process is meta-regularized in order to ensure end-progress. HUND's symmetry between training and testing makes it the first 3d human sensing architecture to natively support different operating regimes including self-supervised ones. In diverse tests, we show that HUND achieves very competitive results in datasets like H3.6M and 3DPW, aswell as good quality 3d reconstructions for complex imagery collected in-the-wild.


THUNDR: Transformer-based 3D HUmaN Reconstruction with Markers

We present THUNDR, a transformer-based deep neural network methodology t...

Weakly Supervised 3D Human Pose and Shape Reconstruction with Normalizing Flows

Monocular 3D human pose and shape estimation is challenging due to the m...

Learning to Regress Bodies from Images using Differentiable Semantic Rendering

Learning to regress 3D human body shape and pose (e.g. SMPL parameters) ...

Learning Transferable Kinematic Dictionary for 3D Human Pose and Shape Reconstruction

Estimating 3D human pose and shape from a single image is highly under-c...

Self-supervised Learning of Motion Capture

Current state-of-the-art solutions for motion capture from a single came...

Appearance Consensus Driven Self-Supervised Human Mesh Recovery

We present a self-supervised human mesh recovery framework to infer huma...