Consider the Alternatives: Navigating Fairness-Accuracy Tradeoffs via Disqualification

by   Guy N. Rothblum, et al.

In many machine learning settings there is an inherent tension between fairness and accuracy desiderata. How should one proceed in light of such trade-offs? In this work we introduce and study γ-disqualification, a new framework for reasoning about fairness-accuracy tradeoffs w.r.t a benchmark class H in the context of supervised learning. Our requirement stipulates that a classifier should be disqualified if it is possible to improve its fairness by switching to another classifier from H without paying "too much" in accuracy. The notion of "too much" is quantified via a parameter γ that serves as a vehicle for specifying acceptable tradeoffs between accuracy and fairness, in a way that is independent from the specific metrics used to quantify fairness and accuracy in a given task. Towards this objective, we establish principled translations between units of accuracy and units of (un)fairness for different accuracy measures. We show γ-disqualification can be used to easily compare different learning strategies in terms of how they trade-off fairness and accuracy, and we give an efficient reduction from the problem of finding the optimal classifier that satisfies our requirement to the problem of approximating the Pareto frontier of H.


page 1

page 2

page 3

page 4


Fairly Accurate: Learning Optimal Accuracy vs. Fairness Tradeoffs for Hate Speech Detection

Recent work has emphasized the importance of balancing competing objecti...

Pareto Efficient Fairness in Supervised Learning: From Extraction to Tracing

As algorithmic decision-making systems are becoming more pervasive, it i...

Accuracy and Fairness Trade-offs in Machine Learning: A Stochastic Multi-Objective Approach

In the application of machine learning to real-life decision-making syst...

The Sharpe predictor for fairness in machine learning

In machine learning (ML) applications, unfair predictions may discrimina...

On the Choice of Fairness: Finding Representative Fairness Metrics for a Given Context

It is of critical importance to be aware of the historical discriminatio...

To the Fairness Frontier and Beyond: Identifying, Quantifying, and Optimizing the Fairness-Accuracy Pareto Frontier

Algorithmic fairness has emerged as an important consideration when usin...

Machine learning for public policy: Do we need to sacrifice accuracy to make models fair?

Growing applications of machine learning in policy settings have raised ...

Please sign up or login with your details

Forgot password? Click here to reset