Scale Invariant Semantic Segmentation with RGB-D Fusion

04/10/2022
by   Mohammad Dawud Ansari, et al.
0

In this paper, we propose a neural network architecture for scale-invariant semantic segmentation using RGB-D images. We utilize depth information as an additional modality apart from color images only. Especially in an outdoor scene which consists of different scale objects due to the distance of the objects from the camera. The near distance objects consist of significantly more pixels than the far ones. We propose to incorporate depth information to the RGB data for pixel-wise semantic segmentation to address the different scale objects in an outdoor scene. We adapt to a well-known DeepLab-v2(ResNet-101) model as our RGB baseline. Depth images are passed separately as an additional input with a distinct branch. The intermediate feature maps of both color and depth image branch are fused using a novel fusion block. Our model is compact and can be easily applied to the other RGB model. We perform extensive qualitative and quantitative evaluation on a challenging dataset Cityscapes. The results obtained are comparable to the state-of-the-art. Additionally, we evaluated our model on a self-recorded real dataset. For the shake of extended evaluation of a driving scene with ground truth we generated a synthetic dataset using popular vehicle simulation project CARLA. The results obtained from the real and synthetic dataset shows the effectiveness of our approach.

READ FULL TEXT

page 2

page 3

page 4

research
01/26/2021

Global-Local Propagation Network for RGB-D Semantic Segmentation

Depth information matters in RGB-D semantic segmentation task for provid...
research
12/25/2019

Multi-Modal Attention-based Fusion Model for Semantic Segmentation of RGB-Depth Images

The 3D scene understanding is mainly considered as a crucial requirement...
research
03/23/2020

Atlas: End-to-End 3D Scene Reconstruction from Posed Images

We present an end-to-end 3D reconstruction method for a scene by directl...
research
09/24/2018

Incorporating Luminance, Depth and Color Information by Fusion-based Networks for Semantic Segmentation

Semantic segmentation is paramount to accomplish many scene understandin...
research
11/26/2020

Polarization-driven Semantic Segmentation via Efficient Attention-bridged Fusion

Semantic Segmentation (SS) is promising for outdoor scene perception in ...
research
08/12/2020

Dynamic Object Removal and Spatio-Temporal RGB-D Inpainting via Geometry-Aware Adversarial Learning

Dynamic objects have a significant impact on the robot's perception of t...
research
07/24/2023

CarPatch: A Synthetic Benchmark for Radiance Field Evaluation on Vehicle Components

Neural Radiance Fields (NeRFs) have gained widespread recognition as a h...

Please sign up or login with your details

Forgot password? Click here to reset