Convolutional Mesh Regression for Single-Image Human Shape Reconstruction

05/08/2019
by   Nikos Kolotouros, et al.
8

This paper addresses the problem of 3D human pose and shape estimation from a single image. Previous approaches consider a parametric model of the human body, SMPL, and attempt to regress the model parameters that give rise to a mesh consistent with image evidence. This parameter regression has been a very challenging task, with model-based approaches underperforming compared to nonparametric solutions in terms of pose estimation. In our work, we propose to relax this heavy reliance on the model's parameter space. We still retain the topology of the SMPL template mesh, but instead of predicting model parameters, we directly regress the 3D location of the mesh vertices. This is a heavy task for a typical network, but our key insight is that the regression becomes significantly easier using a Graph-CNN. This architecture allows us to explicitly encode the template mesh structure within the network and leverage the spatial locality the mesh has to offer. Image-based features are attached to the mesh vertices and the Graph-CNN is responsible to process them on the mesh structure, while the regression target for each vertex is its 3D location. Having recovered the complete 3D geometry of the mesh, if we still require a specific model parametrization, this can be reliably regressed from the vertices locations. We demonstrate the flexibility and the effectiveness of our proposed graph-based mesh regression by attaching different types of features on the mesh vertices. In all cases, we outperform the comparable baselines relying on model parameter regression, while we also achieve state-of-the-art results among model-based pose estimation approaches.

READ FULL TEXT

page 1

page 6

page 7

page 8

research
11/09/2021

Monocular Human Shape and Pose with Dense Mesh-borne Local Image Features

We propose to improve on graph convolution based approaches for human sh...
research
08/27/2021

DC-GNet: Deep Mesh Relation Capturing Graph Convolution Network for 3D Human Shape Reconstruction

In this paper, we aim to reconstruct a full 3D human shape from a single...
research
04/27/2023

A Probabilistic Attention Model with Occlusion-aware Texture Regression for 3D Hand Reconstruction from a Single RGB Image

Recently, deep learning based approaches have shown promising results in...
research
02/28/2020

Learning Nonparametric Human Mesh Reconstruction from a Single Image without Ground Truth Meshes

Nonparametric approaches have shown promising results on reconstructing ...
research
03/09/2020

Hierarchical Kinematic Human Mesh Recovery

We consider the problem of estimating a parametric model of 3D human mes...
research
08/24/2022

Learnable human mesh triangulation for 3D human pose and shape estimation

Compared to joint position, the accuracy of joint rotation and shape est...
research
05/10/2018

Learning to Estimate 3D Human Pose and Shape from a Single Color Image

This work addresses the problem of estimating the full body 3D human pos...

Please sign up or login with your details

Forgot password? Click here to reset