On Self-Contact and Human Pose

04/07/2021
by   Lea Müller, et al.
0

People touch their face 23 times an hour, they cross their arms and legs, put their hands on their hips, etc. While many images of people contain some form of self-contact, current 3D human pose and shape (HPS) regression methods typically fail to estimate this contact. To address this, we develop new datasets and methods that significantly improve human pose estimation with self-contact. First, we create a dataset of 3D Contact Poses (3DCP) containing SMPL-X bodies fit to 3D scans as well as poses from AMASS, which we refine to ensure good contact. Second, we leverage this to create the Mimic-The-Pose (MTP) dataset of images, collected via Amazon Mechanical Turk, containing people mimicking the 3DCP poses with selfcontact. Third, we develop a novel HPS optimization method, SMPLify-XMC, that includes contact constraints and uses the known 3DCP body pose during fitting to create near ground-truth poses for MTP images. Fourth, for more image variety, we label a dataset of in-the-wild images with Discrete Self-Contact (DSC) information and use another new optimization method, SMPLify-DC, that exploits discrete contacts during pose optimization. Finally, we use our datasets during SPIN training to learn a new 3D human pose regressor, called TUCH (Towards Understanding Contact in Humans). We show that the new self-contact training data significantly improves 3D human pose estimates on withheld test data and existing datasets like 3DPW. Not only does our method improve results for self-contact poses, but it also improves accuracy for non-contact poses. The code and data are available for research purposes at https://tuch.is.tue.mpg.de.

READ FULL TEXT

page 1

page 5

page 12

page 13

page 14

page 15

page 17

page 18

research
04/29/2021

AGORA: Avatars in Geography Optimized for Regression Analysis

While the accuracy of 3D human pose estimation from images has steadily ...
research
03/13/2023

PoseExaminer: Automated Testing of Out-of-Distribution Robustness in Human Pose and Shape Estimation

Human pose and shape (HPS) estimation methods achieve remarkable results...
research
03/31/2023

3D Human Pose Estimation via Intuitive Physics

Estimating 3D humans from images often produces implausible bodies that ...
research
12/18/2020

Learning Complex 3D Human Self-Contact

Monocular estimation of three dimensional human self-contact is fundamen...
research
08/03/2023

Reconstructing Three-Dimensional Models of Interacting Humans

Understanding 3d human interactions is fundamental for fine-grained scen...
research
03/27/2023

Hi4D: 4D Instance Segmentation of Close Human Interaction

We propose Hi4D, a method and dataset for the automatic analysis of phys...
research
08/24/2023

POCO: 3D Pose and Shape Estimation with Confidence

The regression of 3D Human Pose and Shape (HPS) from an image is becomin...

Please sign up or login with your details

Forgot password? Click here to reset