Modular Sensor Fusion for Semantic Segmentation

07/30/2018
by   Hermann Blum, et al.
2

Sensor fusion is a fundamental process in robotic systems as it extends the perceptual range and increases robustness in real-world operations. Current multi-sensor deep learning based semantic segmentation approaches do not provide robustness to under-performing classes in one modality, or require a specific architecture with access to the full aligned multi-sensor training data. In this work, we analyze statistical fusion approaches for semantic segmentation that overcome these drawbacks while keeping a competitive performance. The studied approaches are modular by construction, allowing to have different training sets per modality and only a much smaller subset is needed to calibrate the statistical models. We evaluate a range of statistical fusion approaches and report their performance against state-of-the-art baselines on both real-world and simulated data. In our experiments, the approach improves performance in IoU over the best single modality segmentation results by up to 5 available.

READ FULL TEXT

page 2

page 6

research
04/21/2023

Missing Modality Robustness in Semi-Supervised Multi-Modal Semantic Segmentation

Using multiple spatial modalities has been proven helpful in improving s...
research
04/25/2019

Sensor Fusion for Joint 3D Object Detection and Semantic Segmentation

In this paper, we present an extension to LaserNet, an efficient and sta...
research
11/06/2019

UNO: Uncertainty-aware Noisy-Or Multimodal Fusion for Unanticipated Input Degradation

The fusion of multiple sensor modalities, especially through deep learni...
research
05/26/2022

Deep Sensor Fusion with Pyramid Fusion Networks for 3D Semantic Segmentation

Robust environment perception for autonomous vehicles is a tremendous ch...
research
11/20/2018

Sensor Adaptation for Improved Semantic Segmentation of Overhead Imagery

Semantic segmentation is a powerful method to facilitate visual scene un...
research
07/17/2023

Variational Probabilistic Fusion Network for RGB-T Semantic Segmentation

RGB-T semantic segmentation has been widely adopted to handle hard scene...
research
09/02/2020

Multi-domain semantic segmentation with pyramidal fusion

We present our submission to the semantic segmentation contest of the Ro...

Please sign up or login with your details

Forgot password? Click here to reset