HashGAN:Attention-aware Deep Adversarial Hashing for Cross Modal Retrieval

11/26/2017
by   Xi Zhang, et al.
0

As the rapid growth of multi-modal data, hashing methods for cross-modal retrieval have received considerable attention. Deep-networks-based cross-modal hashing methods are appealing as they can integrate feature learning and hash coding into end-to-end trainable frameworks. However, it is still challenging to find content similarities between different modalities of data due to the heterogeneity gap. To further address this problem, we propose an adversarial hashing network with attention mechanism to enhance the measurement of content similarities by selectively focusing on informative parts of multi-modal data. The proposed new adversarial network, HashGAN, consists of three building blocks: 1) the feature learning module to obtain feature representations, 2) the generative attention module to generate an attention mask, which is used to obtain the attended (foreground) and the unattended (background) feature representations, 3) the discriminative hash coding module to learn hash functions that preserve the similarities between different modalities. In our framework, the generative module and the discriminative module are trained in an adversarial way: the generator is learned to make the discriminator cannot preserve the similarities of multi-modal data w.r.t. the background feature representations, while the discriminator aims to preserve the similarities of multi-modal data w.r.t. both the foreground and the background feature representations. Extensive evaluations on several benchmark datasets demonstrate that the proposed HashGAN brings substantial improvements over other state-of-the-art cross-modal hashing methods.

READ FULL TEXT
research
07/29/2019

Deep Cross-Modal Hashing with Hashing Functions and Unified Hash Codes Jointly Learning

Due to their high retrieval efficiency and low storage cost, cross-modal...
research
02/15/2022

Efficient Cross-Modal Retrieval via Deep Binary Hashing and Quantization

Cross-modal retrieval aims to search for data with similar semantic mean...
research
01/20/2022

Deep Unsupervised Contrastive Hashing for Large-Scale Cross-Modal Text-Image Retrieval in Remote Sensing

Due to the availability of large-scale multi-modal data (e.g., satellite...
research
08/22/2023

CLIP Multi-modal Hashing: A new baseline CLIPMH

The multi-modal hashing method is widely used in multimedia retrieval. I...
research
02/26/2022

An Unsupervised Cross-Modal Hashing Method Robust to Noisy Training Image-Text Correspondences in Remote Sensing

The development of accurate and scalable cross-modal image-text retrieva...
research
11/28/2022

Long-tail Cross Modal Hashing

Existing Cross Modal Hashing (CMH) methods are mainly designed for balan...
research
04/04/2018

Discriminative Cross-View Binary Representation Learning

Learning compact representation is vital and challenging for large scale...

Please sign up or login with your details

Forgot password? Click here to reset