Learning Common and Specific Features for RGB-D Semantic Segmentation with Deconvolutional Networks

08/03/2016
by   Jinghua Wang, et al.
0

In this paper, we tackle the problem of RGB-D semantic segmentation of indoor images. We take advantage of deconvolutional networks which can predict pixel-wise class labels, and develop a new structure for deconvolution of multiple modalities. We propose a novel feature transformation network to bridge the convolutional networks and deconvolutional networks. In the feature transformation network, we correlate the two modalities by discovering common features between them, as well as characterize each modality by discovering modality specific features. With the common features, we not only closely correlate the two modalities, but also allow them to borrow features from each other to enhance the representation of shared information. With specific features, we capture the visual patterns that are only visible in one modality. The proposed network achieves competitive segmentation accuracy on NYU depth dataset V1 and V2.

READ FULL TEXT

page 2

page 12

research
12/17/2018

Learning Common Representation from RGB and Depth Images

We propose a new deep learning architecture for the tasks of semantic se...
research
08/24/2023

Channel and Spatial Relation-Propagation Network for RGB-Thermal Semantic Segmentation

RGB-Thermal (RGB-T) semantic segmentation has shown great potential in h...
research
06/29/2019

RFBNet: Deep Multimodal Networks with Residual Fusion Blocks for RGB-D Semantic Segmentation

Signals from RGB and depth data carry complementary information about th...
research
07/17/2020

Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation

Depth information has proven to be a useful cue in the semantic segmenta...
research
06/05/2018

TS-Net: Combining modality specific and common features for multimodal patch matching

Multimodal patch matching addresses the problem of finding the correspon...
research
10/06/2022

Robust Double-Encoder Network for RGB-D Panoptic Segmentation

Perception is crucial for robots that act in real-world environments, as...
research
07/23/2019

Exploring Semantic Segmentation on the DCT Representation

Typical convolutional networks are trained and conducted on RGB images. ...

Please sign up or login with your details

Forgot password? Click here to reset