Hi4D: 4D Instance Segmentation of Close Human Interaction

03/27/2023
by   Yifei Yin, et al.
0

We propose Hi4D, a method and dataset for the automatic analysis of physically close human-human interaction under prolonged contact. Robustly disentangling several in-contact subjects is a challenging task due to occlusions and complex shapes. Hence, existing multi-view systems typically fuse 3D surfaces of close subjects into a single, connected mesh. To address this issue we leverage i) individually fitted neural implicit avatars; ii) an alternating optimization scheme that refines pose and surface through periods of close proximity; and iii) thus segment the fused raw scans into individual instances. From these instances we compile Hi4D dataset of 4D textured scans of 20 subject pairs, 100 sequences, and a total of more than 11K frames. Hi4D contains rich interaction-centric annotations in 2D and 3D alongside accurately registered parametric body models. We define varied human pose and shape estimation tasks on this dataset and provide results from state-of-the-art methods on these benchmarks.

READ FULL TEXT

page 1

page 6

page 8

research
06/20/2022

Capturing and Inferring Dense Full-Body Human-Scene Contact

Inferring human-scene contact (HSC) is the first step toward understandi...
research
12/21/2020

Populating 3D Scenes by Learning Human-Scene Interaction

Humans live within a 3D space and constantly interact with it to perform...
research
04/07/2021

On Self-Contact and Human Pose

People touch their face 23 times an hour, they cross their arms and legs...
research
04/29/2021

AGORA: Avatars in Geography Optimized for Regression Analysis

While the accuracy of 3D human pose estimation from images has steadily ...
research
07/15/2022

Human keypoint detection for close proximity human-robot interaction

We study the performance of state-of-the-art human keypoint detectors in...
research
08/15/2021

U-mesh: Human Correspondence Matching with Mesh Convolutional Networks

The proliferation of 3D scanning technology has driven a need for method...

Please sign up or login with your details

Forgot password? Click here to reset