SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequences

03/27/2021
by   Shun-Cheng Wu, et al.
0

Scene graphs are a compact and explicit representation successfully used in a variety of 2D scene understanding tasks. This work proposes a method to incrementally build up semantic scene graphs from a 3D environment given a sequence of RGB-D frames. To this end, we aggregate PointNet features from primitive scene components by means of a graph neural network. We also propose a novel attention mechanism well suited for partial and missing graph data present in such an incremental reconstruction scenario. Although our proposed method is designed to run on submaps of the scene, we show it also transfers to entire 3D scenes. Experiments show that our approach outperforms 3D scene graph prediction methods by a large margin and its accuracy is on par with other 3D semantic and panoptic segmentation methods while running at 35 Hz.

READ FULL TEXT

page 1

page 3

page 7

page 10

page 14

page 15

page 16

research
05/04/2023

Incremental 3D Semantic Scene Graph Prediction from RGB Sequences

3D semantic scene graphs are a powerful holistic representation as they ...
research
09/13/2021

Incremental Abstraction in Distributed Probabilistic SLAM Graphs

Scene graphs represent the key components of a scene in a compact and se...
research
09/15/2022

Scene Graph Modification as Incremental Structure Expanding

A scene graph is a semantic representation that expresses the objects, a...
research
09/09/2021

Single Image 3D Object Estimation with Primitive Graph Networks

Reconstructing 3D object from a single image (RGB or depth) is a fundame...
research
09/30/2021

Semantic Dense Reconstruction with Consistent Scene Segments

In this paper, a method for dense semantic 3D scene reconstruction from ...
research
08/23/2022

RGB-D Scene Recognition based on Object-Scene Relation

We develop a RGB-D scene recognition model based on object-scene relatio...
research
01/09/2023

SCENE: Reasoning about Traffic Scenes using Heterogeneous Graph Neural Networks

Understanding traffic scenes requires considering heterogeneous informat...

Please sign up or login with your details

Forgot password? Click here to reset