Learning Selective Mutual Attention and Contrast for RGB-D Saliency Detection

10/12/2020
by   Nian Liu, et al.
0

How to effectively fuse cross-modal information is the key problem for RGB-D salient object detection. Early fusion and the result fusion schemes fuse RGB and depth information at the input and output stages, respectively, hence incur the problem of distribution gap or information loss. Many models use the feature fusion strategy but are limited by the low-order point-to-point fusion methods. In this paper, we propose a novel mutual attention model by fusing attention and contexts from different modalities. We use the non-local attention of one modality to propagate long-range contextual dependencies for the other modality, thus leveraging complementary attention cues to perform high-order and trilinear cross-modal interaction. We also propose to induce contrast inference from the mutual attention and obtain a unified model. Considering low-quality depth data may detriment the model performance, we further propose selective attention to reweight the added depth cues. We embed the proposed modules in a two-stream CNN for RGB-D SOD. Experimental results have demonstrated the effectiveness of our proposed model. Moreover, we also construct a new challenging large-scale RGB-D SOD dataset with high-quality, thus can both promote the training and evaluation of deep models.

READ FULL TEXT

page 2

page 4

page 5

page 6

page 8

page 10

page 12

research
09/20/2019

CNN-based RGB-D Salient Object Detection: Learn, Select and Fuse

The goal of this work is to present a systematic solution for RGB-D sali...
research
06/07/2022

Dual Swin-Transformer based Mutual Interactive Network for RGB-D Salient Object Detection

Salient Object Detection is the task of predicting the human attended re...
research
07/03/2023

HODINet: High-Order Discrepant Interaction Network for RGB-D Salient Object Detection

RGB-D salient object detection (SOD) aims to detect the prominent region...
research
03/19/2020

Depth Potentiality-Aware Gated Attention Network for RGB-D Salient Object Detection

There are two main issues in RGB-D salient object detection: (1) how to ...
research
06/22/2022

Depth-aware Glass Surface Detection with Cross-modal Context Mining

Glass surfaces are becoming increasingly ubiquitous as modern buildings ...
research
08/07/2020

Knowing Depth Quality In Advance: A Depth Quality Assessment Method For RGB-D Salient Object Detection

Previous RGB-D salient object detection (SOD) methods have widely adopte...
research
08/01/2019

Two-Stream Video Classification with Cross-Modality Attention

Fusing multi-modality information is known to be able to effectively bri...

Please sign up or login with your details

Forgot password? Click here to reset