Generating Person-Scene Interactions in 3D Scenes

08/12/2020
by   Siwei Zhang, et al.
6

High fidelity digital 3D environments have been proposed in recent years; however, it remains extreme challenging to automatically equip such environment with realistic human bodies. Existing work utilizes images, depths, or semantic maps to represent the scene, and parametric human models to represent 3D bodies in the scene. While being straightforward, their generated human-scene interactions are often lack of naturalness and physical plausibility. Our key observation is that humans interact with the world through body-scene contact. To explicitly and effectively represent the physical contact between the body and the world is essential for modeling human-scene interaction. To that end, we propose a novel interaction representation, which explicitly encodes the proximity between the human body and the 3D scene around it. Specifically, given a set of basis points on a scene mesh, we leverage a conditional variational autoencoder to synthesize the distance from every basis point to its closest point on a human body. The synthesized proximal relationship between the human body and the scene can indicate which region a person tends to contact. Furthermore, based on such synthesized proximity, we can effectively obtain expressive 3D human bodies that naturally interact with the 3D scene. Our perceptual study shows that our model significantly improves the state-of-the-art method, approaching the realism of real human-scene interaction. We believe our method makes an important step towards the fully automatic synthesis of realistic 3D human bodies in 3D scenes. Our code and model will be publicly available for research purpose.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 4

page 5

page 7

page 8

page 14

page 15

page 16

12/21/2020

Populating 3D Scenes by Learning Human-Scene Interaction

Humans live within a 3D space and constantly interact with it to perform...
05/06/2021

Digital Voodoo Dolls

An institution, be it a body of government, commercial enterprise, or a ...
12/05/2019

Generating 3D People in Scenes without People

We present a fully-automatic system that takes a 3D scene and generates ...
12/16/2021

The Wanderings of Odysseus in 3D Scenes

Our goal is to populate digital environments, in which the digital human...
04/09/2018

Binge Watching: Scaling Affordance Learning from Sitcoms

In recent years, there has been a renewed interest in jointly modeling p...
08/18/2021

Stochastic Scene-Aware Motion Prediction

A long-standing goal in computer vision is to capture, model, and realis...
12/10/2015

3D Reconstruction of Crime Scenes and Design Considerations for an Interactive Investigation Tool

Crime Scene Investigation (CSI) is a carefully planned systematic proces...

Code Repositories

PLACE

Official Pytorch implementation for 2020 3DV paper "PLACE: Proximity Learning of Articulation and Contact in 3D Environments" and trained models


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.