Human-centric Indoor Scene Synthesis Using Stochastic Grammar

08/25/2018
by   Siyuan Qi, et al.
0

We present a human-centric method to sample and synthesize 3D room layouts and 2D images thereof, to obtain large-scale 2D/3D image data with perfect per-pixel ground truth. An attributed spatial And-Or graph (S-AOG) is proposed to represent indoor scenes. The S-AOG is a probabilistic grammar model, in which the terminal nodes are object entities. Human contexts as contextual relations are encoded by Markov Random Fields (MRF) on the terminal nodes. We learn the distributions from an indoor scene dataset and sample new layouts using Monte Carlo Markov Chain. Experiments demonstrate that our method can robustly sample a large variety of realistic room layouts based on three criteria: (i) visual realism comparing to a state-of-the-art room arrangement method, (ii) accuracy of the affordance maps with respect to groundtruth, and (ii) the functionality and naturalness of synthesized rooms evaluated by human subjects. The code is available at https://github.com/SiyuanQi/human-centric-scene-synthesis.

READ FULL TEXT
research
08/07/2018

Holistic 3D Scene Parsing and Reconstruction from a Single RGB Image

We propose a computational framework to jointly parse a single RGB image...
research
06/24/2022

HM3D-ABO: A Photo-realistic Dataset for Object-centric Multi-view 3D Reconstruction

Reconstructing 3D objects is an important computer vision task that has ...
research
08/04/2022

360Roam: Real-Time Indoor Roaming Using Geometry-Aware 360^∘ Radiance Fields

Neural radiance field (NeRF) has recently achieved impressive results in...
research
07/21/2022

Human-centric Image Cropping with Partition-aware and Content-preserving Features

Image cropping aims to find visually appealing crops in an image, which ...
research
03/09/2020

Style-compatible Object Recommendation for Multi-room Indoor Scene Synthesis

Traditional indoor scene synthesis methods often take a two-step approac...
research
08/01/2021

Object-to-Scene: Learning to Transfer Object Knowledge to Indoor Scene Recognition

Accurate perception of the surrounding scene is helpful for robots to ma...
research
12/03/2020

Rel3D: A Minimally Contrastive Benchmark for Grounding Spatial Relations in 3D

Understanding spatial relations (e.g., "laptop on table") in visual inpu...

Please sign up or login with your details

Forgot password? Click here to reset