Improving Calibration in Deep Metric Learning With Cross-Example Softmax

11/17/2020
by   Andreas Veit, et al.
0

Modern image retrieval systems increasingly rely on the use of deep neural networks to learn embedding spaces in which distance encodes the relevance between a given query and image. In this setting, existing approaches tend to emphasize one of two properties. Triplet-based methods capture top-k relevancy, where all top-k scoring documents are assumed to be relevant to a given query Pairwise contrastive models capture threshold relevancy, where all documents scoring higher than some threshold are assumed to be relevant. In this paper, we propose Cross-Example Softmax which combines the properties of top-k and threshold relevancy. In each iteration, the proposed loss encourages all queries to be closer to their matching images than all queries are to all non-matching images. This leads to a globally more calibrated similarity metric and makes distance more interpretable as an absolute measure of relevance. We further introduce Cross-Example Negative Mining, in which each pair is compared to the hardest negative comparisons across the entire batch. Empirically, we show in a series of experiments on Conceptual Captions and Flickr30k, that the proposed method effectively improves global calibration and also retrieval performance.

READ FULL TEXT
research
06/15/2021

Hotel Recognition via Latent Image Embedding

We approach the problem of hotel recognition with deep metric learning. ...
research
04/21/2019

Deep Metric Learning Beyond Binary Supervision

Metric Learning for visual similarity has mostly adopted binary supervis...
research
04/14/2017

Cross-media Similarity Metric Learning with Unified Deep Networks

As a highlighting research topic in the multimedia area, cross-media ret...
research
09/05/2017

Cross-Media Similarity Evaluation for Web Image Retrieval in the Wild

In order to retrieve unlabeled images by textual queries, cross-media si...
research
08/20/2021

LoOp: Looking for Optimal Hard Negative Embeddings for Deep Metric Learning

Deep metric learning has been effectively used to learn distance metrics...
research
11/18/2019

Ladder Loss for Coherent Visual-Semantic Embedding

For visual-semantic embedding, the existing methods normally treat the r...
research
05/30/2018

Collaborative Human-AI (CHAI): Evidence-Based Interpretable Melanoma Classification in Dermoscopic Images

Automated dermoscopic image analysis has witnessed rapid growth in diagn...

Please sign up or login with your details

Forgot password? Click here to reset