Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration

04/16/2021
by   Shaofei Wang, et al.
7

Registering point clouds of dressed humans to parametric human models is a challenging task in computer vision. Traditional approaches often rely on heavily engineered pipelines that require accurate manual initialization of human poses and tedious post-processing. More recently, learning-based methods are proposed in hope to automate this process. We observe that pose initialization is key to accurate registration but existing methods often fail to provide accurate pose initialization. One major obstacle is that, regressing joint rotations from point clouds or images of humans is still very challenging. To this end, we propose novel piecewise transformation fields (PTF), a set of functions that learn 3D translation vectors to map any query point in posed space to its correspond position in rest-pose space. We combine PTF with multi-class occupancy networks, obtaining a novel learning-based framework that learns to simultaneously predict shape and per-point correspondences between the posed space and the canonical space for clothed human. Our key insight is that the translation vector for each query point can be effectively estimated using the point-aligned local features; consequently, rigid per bone transformations and joint rotations can be obtained efficiently via a least-square fitting given the estimated point correspondences, circumventing the challenging task of directly regressing joint rotations from neural networks. Furthermore, the proposed PTF facilitate canonicalized occupancy estimation, which greatly improves generalization capability and results in more accurate surface reconstruction with only half of the parameters compared with the state-of-the-art. Both qualitative and quantitative studies show that fitting parametric models with poses initialized by our network results in much better registration quality, especially for extreme poses.

READ FULL TEXT

page 1

page 8

page 13

page 14

research
04/20/2023

Generalizing Neural Human Fitting to Unseen Poses With Articulated SE(3) Equivariance

We address the problem of fitting a parametric human body model (SMPL) t...
research
11/10/2020

Point Cloud Registration Based on Consistency Evaluation of Rigid Transformation in Parameter Space

We can use a method called registration to integrate some point clouds t...
research
04/14/2021

LEAP: Learning Articulated Occupancy of People

Substantial progress has been made on modeling rigid 3D objects using de...
research
07/14/2022

Deep Point-to-Plane Registration by Efficient Backpropagation for Error Minimizing Function

Traditional algorithms of point set registration minimizing point-to-pla...
research
09/21/2022

GNPM: Geometric-Aware Neural Parametric Models

We propose Geometric Neural Parametric Models (GNPM), a learned parametr...
research
11/28/2018

DeepMapping: Unsupervised Map Estimation From Multiple Point Clouds

We propose DeepMapping, a novel registration framework using deep neural...
research
02/18/2023

Invertible Neural Skinning

Building animatable and editable models of clothed humans from raw 3D sc...

Please sign up or login with your details

Forgot password? Click here to reset