CIR-Net: Cross-modality Interaction and Refinement for RGB-D Salient Object Detection

10/06/2022
by   Runmin Cong, et al.
8

Focusing on the issue of how to effectively capture and utilize cross-modality information in RGB-D salient object detection (SOD) task, we present a convolutional neural network (CNN) model, named CIR-Net, based on the novel cross-modality interaction and refinement. For the cross-modality interaction, 1) a progressive attention guided integration unit is proposed to sufficiently integrate RGB-D feature representations in the encoder stage, and 2) a convergence aggregation structure is proposed, which flows the RGB and depth decoding features into the corresponding RGB-D decoding streams via an importance gated fusion unit in the decoder stage. For the cross-modality refinement, we insert a refinement middleware structure between the encoder and the decoder, in which the RGB, depth, and RGB-D encoder features are further refined by successively using a self-modality attention refinement unit and a cross-modality weighting refinement unit. At last, with the gradually refined features, we predict the saliency map in the decoder stage. Extensive experiments on six popular RGB-D SOD benchmarks demonstrate that our network outperforms the state-of-the-art saliency detectors both qualitatively and quantitatively.

READ FULL TEXT

page 1

page 4

page 8

page 11

page 12

page 13

research
07/14/2020

RGB-D Salient Object Detection with Cross-Modality Modulation and Selection

We present an effective method to progressively integrate and refine the...
research
08/17/2023

Point-aware Interaction and CNN-induced Refinement Network for RGB-D Salient Object Detection

By integrating complementary information from RGB image and depth map, t...
research
01/25/2021

RGB-D Salient Object Detection via 3D Convolutional Neural Networks

RGB-D salient object detection (SOD) recently has attracted increasing r...
research
10/10/2021

Modality-Guided Subnetwork for Salient Object Detection

Recent RGBD-based models for saliency detection have attracted research ...
research
04/05/2021

BTS-Net: Bi-directional Transfer-and-Selection Network For RGB-D Salient Object Detection

Depth information has been proved beneficial in RGB-D salient object det...
research
10/12/2022

PSNet: Parallel Symmetric Network for Video Salient Object Detection

For the video salient object detection (VSOD) task, how to excavate the ...
research
10/07/2021

Virtual Multi-Modality Self-Supervised Foreground Matting for Human-Object Interaction

Most existing human matting algorithms tried to separate pure human-only...

Please sign up or login with your details

Forgot password? Click here to reset