SCH-GAN: Semi-supervised Cross-modal Hashing by Generative Adversarial Network

02/07/2018
by   Jian Zhang, et al.
0

Cross-modal hashing aims to map heterogeneous multimedia data into a common Hamming space, which can realize fast and flexible retrieval across different modalities. Supervised cross-modal hashing methods have achieved considerable progress by incorporating semantic side information. However, they mainly have two limitations: (1) Heavily rely on large-scale labeled cross-modal training data which are labor intensive and hard to obtain. (2) Ignore the rich information contained in the large amount of unlabeled data across different modalities, especially the margin examples that are easily to be incorrectly retrieved, which can help to model the correlations. To address these problems, in this paper we propose a novel Semi-supervised Cross-Modal Hashing approach by Generative Adversarial Network (SCH-GAN). We aim to take advantage of GAN's ability for modeling data distributions to promote cross-modal hashing learning in an adversarial way. The main contributions can be summarized as follows: (1) We propose a novel generative adversarial network for cross-modal hashing. In our proposed SCH-GAN, the generative model tries to select margin examples of one modality from unlabeled data when giving a query of another modality. While the discriminative model tries to distinguish the selected examples and true positive examples of the query. These two models play a minimax game so that the generative model can promote the hashing performance of discriminative model. (2) We propose a reinforcement learning based algorithm to drive the training of proposed SCH-GAN. The generative model takes the correlation score predicted by discriminative model as a reward, and tries to select the examples close to the margin to promote discriminative model by maximizing the margin between positive and negative data. Experiments on 3 widely-used datasets verify the effectiveness of our proposed approach.

READ FULL TEXT

page 1

page 4

research
12/01/2017

Unsupervised Generative Adversarial Cross-modal Hashing

Cross-modal hashing aims to map heterogeneous multimedia data into a com...
research
10/29/2017

A Novel Approach to Artistic Textual Visualization via GAN

While the visualization of statistical data tends to a mature technology...
research
04/02/2018

SyncGAN: Synchronize the Latent Space of Cross-modal Generative Adversarial Networks

Generative adversarial network (GAN) has achieved impressive success on ...
research
11/20/2019

SSAH: Semi-supervised Adversarial Deep Hashing with Self-paced Hard Sample Generation

Deep hashing methods have been proved to be effective and efficient for ...
research
05/27/2019

Label Prediction Framework for Semi-Supervised Cross-Modal Retrieval

Cross-modal data matching refers to retrieval of data from one modality,...
research
02/04/2023

TAP: The Attention Patch for Cross-Modal Knowledge Transfer from Unlabeled Data

This work investigates the intersection of cross modal learning and semi...
research
07/02/2020

A Semi-Supervised Generative Adversarial Network for Prediction of Genetic Disease Outcomes

For most diseases, building large databases of labeled genetic data is a...

Please sign up or login with your details

Forgot password? Click here to reset