Label Prediction Framework for Semi-Supervised Cross-Modal Retrieval

05/27/2019
by   Devraj Mandal, et al.
0

Cross-modal data matching refers to retrieval of data from one modality, when given a query from another modality. In general, supervised algorithms achieve better retrieval performance compared to their unsupervised counterpart, as they can learn better representative features by leveraging the available label information. However, this comes at the cost of requiring huge amount of labeled examples, which may not always be available. In this work, we propose a novel framework in a semi-supervised setting, which can predict the labels of the unlabeled data using complementary information from different modalities. The proposed framework can be used as an add-on with any baseline crossmodal algorithm to give significant performance improvement, even in case of limited labeled data. Finally, we analyze the challenging scenario where the unlabeled examples can even come from classes not in the training data and evaluate the performance of our algorithm under such setting. Extensive evaluation using several baseline algorithms across three different datasets shows the effectiveness of our label prediction framework.

READ FULL TEXT

page 1

page 4

research
12/04/2018

A Deep Learning Framework for Semi-Supervised Cross-Modal Retrieval with Label Prediction

Due to abundance of data from multiple modalities, cross-modal retrieval...
research
03/21/2017

Cross-modal Deep Metric Learning with Multi-task Regularization

DNN-based cross-modal retrieval has become a research hotspot, by which ...
research
02/04/2023

TAP: The Attention Patch for Cross-Modal Knowledge Transfer from Unlabeled Data

This work investigates the intersection of cross modal learning and semi...
research
02/07/2018

SCH-GAN: Semi-supervised Cross-modal Hashing by Generative Adversarial Network

Cross-modal hashing aims to map heterogeneous multimedia data into a com...
research
08/12/2021

Trash to Treasure: Harvesting OOD Data with Cross-Modal Matching for Open-Set Semi-Supervised Learning

Open-set semi-supervised learning (open-set SSL) investigates a challeng...
research
12/09/2017

Semi-supervised Multimodal Hashing

Retrieving nearest neighbors across correlated data in multiple modaliti...
research
09/13/2023

Reliability-based cleaning of noisy training labels with inductive conformal prediction in multi-modal biomedical data mining

Accurately labeling biomedical data presents a challenge. Traditional se...

Please sign up or login with your details

Forgot password? Click here to reset