Quantitative robustness of instance ranking problems

03/12/2021
by   Tino Werner, et al.
0

Instance ranking problems intend to recover the true ordering of the instances in a data set with a variety of applications in for example scientific, social and financial contexts. Robust statistics studies the behaviour of estimators in the presence of perturbations of the data resp. the underlying distribution and provides different concepts to characterize local and global robustness. In this work, we concentrate on the global robustness of parametric ranking problems in terms of the breakdown point which measures the fraction of samples that need to be perturbed in order to let the estimator take unreasonable values. However, existing breakdown point notions do not cover ranking problems so far. We propose to define a breakdown of the estimator as a sign-reversal of all components which causes the predicted ranking to be inverted, therefore we call our concept the order-inversal breakdown point (OIBDP). We will study the OIBDP, based on a linear model, for several different ranking problems that we carefully distinguish and provide least favorable outlier configurations, characterizations of the order-inversal breakdown point as well as sharp asymptotic upper bounds. We also outline the case of SVM-type ranking estimators.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/12/2018

Ranking Robustness Under Adversarial Document Manipulations

For many queries in the Web retrieval setting there is an on-going ranki...
research
03/22/2023

Robust Consensus in Ranking Data Analysis: Definitions, Properties and Computational Issues

As the issue of robustness in AI systems becomes vital, statistical lear...
research
09/06/2019

A review on ranking problems in statistical learning

Ranking problems define a widely spread class of statistical learning pr...
research
09/05/2016

The Robustness of Estimator Composition

We formalize notions of robustness for composite estimators via the noti...
research
08/11/2021

Are Neural Ranking Models Robust?

Recently, we have witnessed the bloom of neural ranking models in the in...
research
04/24/2013

A Theoretical Analysis of NDCG Type Ranking Measures

A central problem in ranking is to design a ranking measure for evaluati...
research
06/24/2021

On Locality of Local Explanation Models

Shapley values provide model agnostic feature attributions for model out...

Please sign up or login with your details

Forgot password? Click here to reset