Semi-Supervised Cross-Modal Salient Object Detection with U-Structure Networks

08/08/2022
by   Yunqing Bao, et al.
9

Salient Object Detection (SOD) is a popular and important topic aimed at precise detection and segmentation of the interesting regions in the images. We integrate the linguistic information into the vision-based U-Structure networks designed for salient object detection tasks. The experiments are based on the newly created DUTS Cross Modal (DUTS-CM) dataset, which contains both visual and linguistic labels. We propose a new module called efficient Cross-Modal Self-Attention (eCMSA) to combine visual and linguistic features and improve the performance of the original U-structure networks. Meanwhile, to reduce the heavy burden of labeling, we employ a semi-supervised learning method by training an image caption model based on the DUTS-CM dataset, which can automatically label other datasets like DUT-OMRON and HKU-IS. The comprehensive experiments show that the performance of SOD can be improved with the natural language input and is competitive compared with other SOD methods.

READ FULL TEXT

page 1

page 4

page 5

page 7

page 9

research
01/24/2022

Multi-Scale Iterative Refinement Network for RGB-D Salient Object Detection

The extensive research leveraging RGB-D information has been exploited i...
research
02/16/2023

Hierarchical Cross-modal Transformer for RGB-D Salient Object Detection

Most of existing RGB-D salient object detection (SOD) methods follow the...
research
01/31/2019

Self-Supervised Visual Representations for Cross-Modal Retrieval

Cross-modal retrieval methods have been significantly improved in last y...
research
04/09/2019

Cross-Modal Self-Attention Network for Referring Image Segmentation

We consider the problem of referring image segmentation. Given an input ...
research
02/09/2021

Referring Segmentation in Images and Videos with Cross-Modal Self-Attention Network

We consider the problem of referring segmentation in images and videos w...
research
08/07/2020

A Novel Video Salient Object Detection Method via Semi-supervised Motion Quality Perception

Previous video salient object detection (VSOD) approaches have mainly fo...
research
08/06/2021

Full-Duplex Strategy for Video Object Segmentation

Appearance and motion are two important sources of information in video ...

Please sign up or login with your details

Forgot password? Click here to reset