DeepAI AI Chat
Log In Sign Up

Stretching the Effectiveness of MLE from Accuracy to Bias for Pairwise Comparisons

by   Jingyan Wang, et al.
Carnegie Mellon University

A number of applications (e.g., AI bot tournaments, sports, peer grading, crowdsourcing) use pairwise comparison data and the Bradley-Terry-Luce (BTL) model to evaluate a given collection of items (e.g., bots, teams, students, search results). Past work has shown that under the BTL model, the widely-used maximum-likelihood estimator (MLE) is minimax-optimal in estimating the item parameters, in terms of the mean squared error. However, another important desideratum for designing estimators is fairness. In this work, we consider fairness modeled by the notion of bias in statistics. We show that the MLE incurs a suboptimal rate in terms of bias. We then propose a simple modification to the MLE, which "stretches" the bounding box of the maximum-likelihood optimizer by a small constant factor from the underlying ground truth domain. We show that this simple modification leads to an improved rate in bias, while maintaining minimax-optimality in the mean squared error. In this manner, our proposed class of estimators provably improves fairness represented by bias without loss in accuracy.


Rao-Blackwellizing the Straight-Through Gumbel-Softmax Gradient Estimator

Gradient estimation in models with discrete latent variables is a challe...

Maximum likelihood quantum state tomography is inadmissible

Maximum likelihood estimation (MLE) is the most common approach to quant...

Efficient Estimation of Smooth Functionals in Gaussian Shift Models

We study a problem of estimation of smooth functionals of parameter θ o...

POT-flavored estimator of Pickands dependence function

This work proposes an estimator with both Peak-Over-Threshold and Block-...

Towards Prevention of Sportsmen Burnout: Formal Analysis of Sub-Optimal Tournament Scheduling

Scheduling a sports tournament is a complex optimization problem, which ...

The bias of isotonic regression

We study the bias of the isotonic regression estimator. While there is e...

Minimax Estimation of Quadratic Fourier Functionals

We study estimation of (semi-)inner products between two nonparametric p...