DeepAI
Log In Sign Up

Learning Fair Scoring Functions: Fairness Definitions, Algorithms and Generalization Bounds for Bipartite Ranking

02/19/2020
by   Robin Vogel, et al.
0

Many applications of artificial intelligence, ranging from credit lending to the design of medical diagnosis support tools through recidivism prediction, involve scoring individuals using a learned function of their attributes. These predictive risk scores are used to rank a set of people, and/or take individual decisions about them based on whether the score exceeds a certain threshold that may depend on the context in which the decision is taken. The level of delegation granted to such systems will heavily depend on how questions of fairness can be answered. While this concern has received a lot of attention in the classification setup, the design of relevant fairness constraints for the problem of learning scoring functions has not been much investigated. In this paper, we propose a flexible approach to group fairness for the scoring problem with binary labeled data, a standard learning task referred to as bipartite ranking. We argue that the functional nature of the ROC curve, the gold standard measuring ranking performance in this context, leads to several possible ways of formulating fairness constraints. We introduce general classes of fairness conditions in bipartite ranking and establish generalization bounds for scoring rules learned under such constraints. Beyond the theoretical formulation and results, we design practical learning algorithms and illustrate our approach with numerical experiments.

READ FULL TEXT

page 25

page 26

02/15/2019

The Fairness of Risk Scores Beyond Classification: Bipartite Ranking and the xAUC Metric

Where machine-learned predictive risk scores inform high-stakes decision...
04/07/2021

Concentration Inequalities for Two-Sample Rank Processes with Application to Bipartite Ranking

The ROC curve is the gold standard for measuring the performance of a te...
12/18/2013

Functional Bipartite Ranking: a Wavelet-Based Filtering Approach

It is the main goal of this article to address the bipartite ranking iss...
11/22/2019

Responsible Scoring Mechanisms Through Function Sampling

Human decision-makers often receive assistance from data-driven algorith...
11/02/2018

The age of secrecy and unfairness in recidivism prediction

In our current society, secret algorithms make important decisions about...
11/09/2015

PAC-Bayesian High Dimensional Bipartite Ranking

This paper is devoted to the bipartite ranking problem, a classical stat...
06/22/2022

FairGrad: Fairness Aware Gradient Descent

We tackle the problem of group fairness in classification, where the obj...