Depth Quality-Inspired Feature Manipulation for Efficient RGB-D Salient Object Detection

07/05/2021
by   Wenbo Zhang, et al.
0

RGB-D salient object detection (SOD) recently has attracted increasing research interest by benefiting conventional RGB SOD with extra depth information. However, existing RGB-D SOD models often fail to perform well in terms of both efficiency and accuracy, which hinders their potential applications on mobile devices and real-world problems. An underlying challenge is that the model accuracy usually degrades when the model is simplified to have few parameters. To tackle this dilemma and also inspired by the fact that depth quality is a key factor influencing the accuracy, we propose a novel depth quality-inspired feature manipulation (DQFM) process, which is efficient itself and can serve as a gating mechanism for filtering depth features to greatly boost the accuracy. DQFM resorts to the alignment of low-level RGB and depth features, as well as holistic attention of the depth stream to explicitly control and enhance cross-modal fusion. We embed DQFM to obtain an efficient light-weight model called DFM-Net, where we also design a tailored depth backbone and a two-stage decoder for further efficiency consideration. Extensive experimental results demonstrate that our DFM-Net achieves state-of-the-art accuracy when comparing to existing non-efficient models, and meanwhile runs at 140ms on CPU (2.2× faster than the prior fastest efficient model) with only ∼8.5Mb model size (14.9 lightest). Our code will be available at https://github.com/zwbx/DFM-Net.

READ FULL TEXT

page 1

page 3

page 4

page 5

page 7

research
08/08/2022

Depth Quality-Inspired Feature Manipulation for Efficient RGB-D and Video Salient Object Detection

Recently CNN-based RGB-D salient object detection (SOD) has obtained sig...
research
01/25/2021

RGB-D Salient Object Detection via 3D Convolutional Neural Networks

RGB-D salient object detection (SOD) recently has attracted increasing r...
research
11/11/2020

FINO-Net: A Deep Multimodal Sensor Fusion Framework for Manipulation Failure Detection

Safe manipulation in unstructured environments for service robots is a c...
research
09/18/2023

DFormer: Rethinking RGBD Representation Learning for Semantic Segmentation

We present DFormer, a novel RGB-D pretraining framework to learn transfe...
research
08/26/2020

Siamese Network for RGB-D Salient Object Detection and Beyond

Existing RGB-D salient object detection (SOD) models usually treat RGB a...
research
03/09/2022

Fast Road Segmentation via Uncertainty-aware Symmetric Network

The high performance of RGB-D based road segmentation methods contrasts ...
research
08/07/2020

Knowing Depth Quality In Advance: A Depth Quality Assessment Method For RGB-D Salient Object Detection

Previous RGB-D salient object detection (SOD) methods have widely adopte...

Please sign up or login with your details

Forgot password? Click here to reset