Putting Humans in a Scene: Learning Affordance in 3D Indoor Environments

03/13/2019
by   Xueting Li, et al.
12

Affordance modeling plays an important role in visual understanding. In this paper, we aim to predict affordances of 3D indoor scenes, specifically what human poses are afforded by a given indoor environment, such as sitting on a chair or standing on the floor. In order to predict valid affordances and learn possible 3D human poses in indoor scenes, we need to understand the semantic and geometric structure of a scene as well as its potential interactions with a human. To learn such a model, a large-scale dataset of 3D indoor affordances is required. In this work, we build a fully automatic 3D pose synthesizer that fuses semantic knowledge from a large number of 2D poses extracted from TV shows as well as 3D geometric knowledge from voxel representations of indoor scenes. With the data created by the synthesizer, we introduce a 3D pose generative model to predict semantically plausible and physically feasible human poses within a given scene (provided as a single RGB, RGB-D, or depth image). We demonstrate that our human affordance prediction method consistently outperforms existing state-of-the-art methods.

READ FULL TEXT

page 1

page 6

page 7

page 8

page 9

page 11

page 13

page 14

research
12/02/2020

Holistic 3D Human and Scene Mesh Estimation from Single View Images

The 3D world limits the human body pose and the human body pose conveys ...
research
07/28/2022

The One Where They Reconstructed 3D Humans and Environments in TV Shows

TV shows depict a wide variety of human behaviors and have been studied ...
research
08/10/2018

Weakly supervised learning of indoor geometry by dual warping

A major element of depth perception and 3D understanding is the ability ...
research
02/22/2020

Shallow2Deep: Indoor Scene Modeling by Single Image Understanding

Dense indoor scene modeling from 2D images has been bottlenecked due to ...
research
06/29/2017

Analysis and Modeling of 3D Indoor Scenes

We live in a 3D world, performing activities and interacting with object...
research
04/14/2020

Footprints and Free Space from a Single Color Image

Understanding the shape of a scene from a single color image is a formid...
research
01/21/2020

Geometric Proxies for Live RGB-D Stream Enhancement and Consolidation

We propose a geometric superstructure for unified real-time processing o...

Please sign up or login with your details

Forgot password? Click here to reset