MCTS with Refinement for Proposals Selection Games in Scene Understanding

07/07/2022
by   Sinisa Stekovic, et al.
11

We propose a novel method applicable in many scene understanding problems that adapts the Monte Carlo Tree Search (MCTS) algorithm, originally designed to learn to play games of high-state complexity. From a generated pool of proposals, our method jointly selects and optimizes proposals that minimize the objective term. In our first application for floor plan reconstruction from point clouds, our method selects and refines the room proposals, modelled as 2D polygons, by optimizing on an objective function combining the fitness as predicted by a deep network and regularizing terms on the room shapes. We also introduce a novel differentiable method for rendering the polygonal shapes of these proposals. Our evaluations on the recent and challenging Structured3D and Floor-SP datasets show significant improvements over the state-of-the-art, without imposing hard constraints nor assumptions on the floor plan configurations. In our second application, we extend our approach to reconstruct general 3D room layouts from a color image and obtain accurate room layouts. We also show that our differentiable renderer can easily be extended for rendering 3D planar polygons and polygon embeddings. Our method shows high performance on the Matterport3D-Layout dataset, without introducing hard constraints on room layout configurations.

READ FULL TEXT

page 1

page 7

page 10

page 12

page 14

research
03/20/2021

MonteFloor: Extending MCTS for Reconstructing Accurate Large-Scale Floor Plans

We propose a novel method for reconstructing floor plans from noisy 3D p...
research
03/14/2021

Monte Carlo Scene Search for 3D Scene Understanding

We explore how a general AI algorithm can be used for 3D scene understan...
research
01/07/2020

General 3D Room Layout from a Single View by Render-and-Compare

We present a novel method to reconstruct the 3D layout of a room – walls...
research
12/12/2021

360-DFPE: Leveraging Monocular 360-Layouts for Direct Floor Plan Estimation

We present 360-DFPE, a sequential floor plan estimation method that dire...
research
08/07/2018

Holistic 3D Scene Parsing and Reconstruction from a Single RGB Image

We propose a computational framework to jointly parse a single RGB image...
research
09/12/2021

PQ-Transformer: Jointly Parsing 3D Objects and Layouts from Point Clouds

3D scene understanding from point clouds plays a vital role for various ...

Please sign up or login with your details

Forgot password? Click here to reset