Practical Compositional Fairness: Understanding Fairness in Multi-Task ML Systems

11/05/2019
by   Xuezhi Wang, et al.
23

Most literature in fairness has focused on improving fairness with respect to one single model or one single objective. However, real-world machine learning systems are usually composed of many different components. Unfortunately, recent research has shown that even if each component is “fair,” the overall system can still be “unfair”<cit.>. In this paper, we focus on how well fairness composes over multiple components in real systems. We consider two recently proposed fairness metrics for rankings: exposure and pairwise ranking accuracy gap. We provide theory that demonstrates a set of conditions under which fairness of individual models does compose. We then present an analytical framework for both understanding whether a system's signals can achieve compositional fairness, and diagnosing which of these signals lowers the overall system's end-to-end fairness the most. Despite previously bleak theoretical results, on multiple data-sets—including a large-scale real-world recommender system—we find that the overall system's end-to-end fairness is largely achievable by improving fairness in individual components.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/26/2023

Insights From Insurance for Fair Machine Learning: Responsibility, Performativity and Aggregates

We argue that insurance can act as an analogon for the social situatedne...
research
07/11/2023

Towards A Scalable Solution for Improving Multi-Group Fairness in Compositional Classification

Despite the rich literature on machine learning fairness, relatively lit...
research
03/02/2019

Fairness in Recommendation Ranking through Pairwise Comparisons

Recommender systems are one of the most pervasive applications of machin...
research
06/25/2020

SenSeI: Sensitive Set Invariance for Enforcing Individual Fairness

In this paper, we cast fair machine learning as invariant machine learni...
research
10/25/2019

Toward a better trade-off between performance and fairness with kernel-based distribution matching

As recent literature has demonstrated how classifiers often carry uninte...
research
06/23/2020

A Framework for Fairness in Two-Sided Marketplaces

Many interesting problems in the Internet industry can be framed as a tw...
research
04/08/2022

Individually-Fair Auctions for Multi-Slot Sponsored Search

We design fair sponsored search auctions that achieve a near-optimal tra...

Please sign up or login with your details

Forgot password? Click here to reset