Robust Double-Encoder Network for RGB-D Panoptic Segmentation

10/06/2022
by   Matteo Sodano, et al.
0

Perception is crucial for robots that act in real-world environments, as autonomous systems need to see and understand the world around them to act appropriately. Panoptic segmentation provides an interpretation of the scene by computing a pixel-wise semantic label together with instance IDs. In this paper, we address panoptic segmentation using RGB-D data of indoor scenes. We propose a novel encoder-decoder neural network that processes RGB and depth separately through two encoders. The features of the individual encoders are progressively merged at different resolutions, such that the RGB features are enhanced using complementary depth information. We propose a novel merging approach called ResidualExcite, which reweighs each entry of the feature map according to its importance. With our double-encoder architecture, we are robust to missing cues. In particular, the same model can train and infer on RGB-D, RGB-only, and depth-only input data, without the need to train specialized models. We evaluate our method on publicly available datasets and show that our approach achieves superior results compared to other common approaches for panoptic segmentation.

READ FULL TEXT

page 1

page 2

page 5

research
06/04/2018

RedNet: Residual Encoder-Decoder Network for indoor RGB-D Semantic Segmentation

Indoor semantic segmentation has always been a difficult task in compute...
research
06/08/2023

Efficient Multi-Task Scene Analysis with RGB-D Transformers

Scene analysis is essential for enabling autonomous systems, such as mob...
research
12/17/2018

Learning Common Representation from RGB and Depth Images

We propose a new deep learning architecture for the tasks of semantic se...
research
06/29/2019

RFBNet: Deep Multimodal Networks with Residual Fusion Blocks for RGB-D Semantic Segmentation

Signals from RGB and depth data carry complementary information about th...
research
08/03/2016

Learning Common and Specific Features for RGB-D Semantic Segmentation with Deconvolutional Networks

In this paper, we tackle the problem of RGB-D semantic segmentation of i...
research
09/06/2015

Joint Color-Spatial-Directional clustering and Region Merging (JCSD-RM) for unsupervised RGB-D image segmentation

Recent advances in depth imaging sensors provide easy access to the sync...
research
08/20/2021

Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation in Real-World Navigation Assistance

Transparent objects, such as glass walls and doors, constitute architect...

Please sign up or login with your details

Forgot password? Click here to reset