DeepAI AI Chat
Log In Sign Up

Delving Deep into Pixel Alignment Feature for Accurate Multi-view Human Mesh Recovery

01/15/2023
by   Kai Jia, et al.
University of Michigan
Tsinghua University
0

Regression-based methods have shown high efficiency and effectiveness for multi-view human mesh recovery. The key components of a typical regressor lie in the feature extraction of input views and the fusion of multi-view features. In this paper, we present Pixel-aligned Feedback Fusion (PaFF) for accurate yet efficient human mesh recovery from multi-view images. PaFF is an iterative regression framework that performs feature extraction and fusion alternately. At each iteration, PaFF extracts pixel-aligned feedback features from each input view according to the reprojection of the current estimation and fuses them together with respect to each vertex of the downsampled mesh. In this way, our regressor can not only perceive the misalignment status of each view from the feedback features but also correct the mesh parameters more effectively based on the feature fusion on mesh vertices. Additionally, our regressor disentangles the global orientation and translation of the body mesh from the estimation of mesh parameters such that the camera parameters of input views can be better utilized in the regression process. The efficacy of our method is validated in the Human3.6M dataset via comprehensive ablation experiments, where PaFF achieves 33.02 MPJPE and brings significant improvements over the previous best solutions by more than 29 results can be found at https://kairobo.github.io/PaFF/.

READ FULL TEXT

page 3

page 4

page 6

page 7

10/04/2022

Multi-view Human Body Mesh Translator

Existing methods for human mesh recovery mainly focus on single-view fra...
07/13/2022

PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images

We present PyMAF-X, a regression-based approach to recovering a full-bod...
04/08/2023

POEM: Reconstructing Hand in a Point Embedded Multi-view Stereo

Enable neural networks to capture 3D geometrical-aware features is essen...
03/30/2021

3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop

Regression-based methods have recently shown promising results in recons...
12/01/2020

Facetwise Mesh Refinement for Multi-View Stereo

Mesh refinement is a fundamental step for accurate Multi-View Stereo. It...
12/10/2022

Progressive Multi-view Human Mesh Recovery with Self-Supervision

To date, little attention has been given to multi-view 3D human mesh est...
08/13/2020

3D Bird Reconstruction: a Dataset, Model, and Shape Recovery from a Single View

Automated capture of animal pose is transforming how we study neuroscien...