Optimizing Rank-based Metrics with Blackbox Differentiation

12/07/2019
by   Michal Rolinek, et al.
18

Rank-based metrics are some of the most widely used criteria for performance evaluation of computer vision models. Despite years of effort, direct optimization for these metrics remains a challenge due to their non-differentiable and non-decomposable nature. We present an efficient, theoretically sound, and general method for differentiating rank-based metrics with mini-batch gradient descent. In addition, we address optimization instability and sparsity of the supervision signal that both arise from using rank-based metrics as optimization targets. Resulting losses based on recall and Average Precision are applied to image retrieval and object detection tasks. We obtain performance that is competitive with state-of-the-art on standard image retrieval datasets and consistently improve performance of near state-of-the-art object detectors.

READ FULL TEXT

page 1

page 8

page 16

page 17

research
09/15/2023

Optimization of Rank Losses for Image Retrieval

In image retrieval, standard evaluation metrics rely on score ranking, a...
research
08/07/2017

Training Deep Networks to be Spatially Sensitive

In many computer vision tasks, for example saliency prediction or semant...
research
03/17/2023

IRGen: Generative Modeling for Image Retrieval

While generative modeling has been ubiquitous in natural language proces...
research
08/31/2020

Optimize What You Evaluate With: A Simple Yet Effective Framework For Direct Optimization Of IR Metrics

Learning-to-rank has been intensively studied and has shown significantl...
research
04/21/2021

MetricOpt: Learning to Optimize Black-Box Evaluation Metrics

We study the problem of directly optimizing arbitrary non-differentiable...
research
08/08/2020

How Trustworthy are the Existing Performance Evaluations for Basic Vision Tasks?

Performance evaluation is indispensable to the advancement of machine vi...
research
05/03/2021

SmoothI: Smooth Rank Indicators for Differentiable IR Metrics

Information retrieval (IR) systems traditionally aim to maximize metrics...

Please sign up or login with your details

Forgot password? Click here to reset