A Generative Model of 3D Object Layouts in Apartments

11/29/2017
by   Paul Henderson, et al.
0

Understanding indoor scenes is an important task in computer vision. This task is typically ambiguous, so we require a strong prior, that captures the regularity of indoor environments. This is naturally expressed by a probabilistic model over 3D room layouts and geometry, reasoning over complex layouts in 3D space, including high-order spatial relations among objects. In this work, we construct such a model, trained on over 250000 human-designed rooms with 170 object classes. We conduct extensive experiments to show the quality of our model. First, we show that it generates samples that are plausible, by an extensive user study involving human comparisons of sampled layouts to ground-truth. Second, we demonstrate the value of incorporating spatial relationships between objects, by showing that this increases the plausibility of samples. Third, we show that our model generalises, rather than simply memorising its training set. Finally, we provide many examples of knowledge learnt by our model, such as support relationships, and common spatial relations between object classes.

READ FULL TEXT

page 1

page 4

page 6

page 7

page 8

research
03/31/2023

Grounding Object Relations in Language-Conditioned Robotic Manipulation with Semantic-Spatial Reasoning

Grounded understanding of natural language in physical scenes can greatl...
research
06/27/2012

Learning Object Arrangements in 3D Scenes using Human Context

We consider the problem of learning object arrangements in a 3D scene. T...
research
12/10/2021

IFR-Explore: Learning Inter-object Functional Relationships in 3D Indoor Scenes

Building embodied intelligent agents that can interact with 3D indoor en...
research
07/19/2020

Understanding Spatial Relations through Multiple Modalities

Recognizing spatial relations and reasoning about them is essential in m...
research
11/17/2022

Language Conditioned Spatial Relation Reasoning for 3D Object Grounding

Localizing objects in 3D scenes based on natural language requires under...
research
11/22/2011

Contextually Guided Semantic Labeling and Search for 3D Point Clouds

RGB-D cameras, which give an RGB image to- gether with depths, are becom...
research
08/07/2019

SpatialSense: An Adversarially Crowdsourced Benchmark for Spatial Relation Recognition

Understanding the spatial relations between objects in images is a surpr...

Please sign up or login with your details

Forgot password? Click here to reset