Semi-Supervised Semantic Mapping through Label Propagation with Semantic Texture Meshes

06/17/2019
by   Radu Alexandru Rosu, et al.
4

Scene understanding is an important capability for robots acting in unstructured environments. While most SLAM approaches provide a geometrical representation of the scene, a semantic map is necessary for more complex interactions with the surroundings. Current methods treat the semantic map as part of the geometry which limits scalability and accuracy. We propose to represent the semantic map as a geometrical mesh and a semantic texture coupled at independent resolution. The key idea is that in many environments the geometry can be greatly simplified without loosing fidelity, while semantic information can be stored at a higher resolution, independent of the mesh. We construct a mesh from depth sensors to represent the scene geometry and fuse information into the semantic texture from segmentations of individual RGB views of the scene. Making the semantics persistent in a global mesh enables us to enforce temporal and spatial consistency of the individual view predictions. For this, we propose an efficient method of establishing consensus between individual segmentations by iteratively retraining semantic segmentation with the information stored within the map and using the retrained segmentation to re-fuse the semantics. We demonstrate the accuracy and scalability of our approach by reconstructing semantic maps of scenes from NYUv2 and a scene spanning large buildings.

READ FULL TEXT

page 3

page 8

page 12

page 14

page 16

research
07/26/2020

Virtual Multi-view Fusion for 3D Semantic Segmentation

Semantic segmentation of 3D meshes is an important problem for 3D scene ...
research
12/27/2018

S4-Net: Geometry-Consistent Semi-Supervised Semantic Segmentation

We show that it is possible to learn semantic segmentation from very lim...
research
04/24/2018

MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects

We present MaskFusion, a real-time, object-aware, semantic and dynamic R...
research
09/15/2021

S3LAM: Structured Scene SLAM

We propose a new general SLAM system that uses the semantic segmentation...
research
10/06/2019

3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera

A comprehensive semantic understanding of a scene is important for many ...
research
02/24/2020

Comparing View-Based and Map-Based Semantic Labelling in Real-Time SLAM

Generally capable Spatial AI systems must build persistent scene represe...
research
09/23/2021

Semi-automatic conversion from OSG to CityGML

CityGML is a data model used to represent the geometric and semantic inf...

Please sign up or login with your details

Forgot password? Click here to reset