Model-Agnostic Characterization of Fairness Trade-offs

04/07/2020
by   Joon Sik Kim, et al.
0

There exist several inherent trade-offs in designing a fair model, such as those between the model's predictive performance and fairness, or even among different notions of fairness. In practice, exploring these trade-offs requires significant human and computational resources. We propose a diagnostic that enables practitioners to explore these trade-offs without training a single model. Our work hinges on the observation that many widely-used fairness definitions can be expressed via the fairness-confusion tensor, an object obtained by splitting the traditional confusion matrix according to protected data attributes. Optimizing accuracy and fairness objectives directly over the elements in this tensor yields a data-dependent yet model-agnostic way of understanding several types of trade-offs. We further leverage this tensor-based perspective to generalize existing theoretical impossibility results to a wider range of fairness definitions. Finally, we demonstrate the usefulness of the proposed diagnostic on synthetic and real datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/26/2022

SoFaiR: Single Shot Fair Representation Learning

To avoid discriminatory uses of their data, organizations can learn to m...
research
06/30/2020

Evaluation of Fairness Trade-offs in Predicting Student Success

Predictive models for identifying at-risk students early can help teachi...
research
02/24/2022

Trade-offs between Group Fairness Metrics in Societal Resource Allocation

We consider social resource allocations that deliver an array of scarce ...
research
09/19/2016

Inherent Trade-Offs in the Fair Determination of Risk Scores

Recent discussion in the public sphere about algorithmic classification ...
research
09/13/2021

On the Choice of Fairness: Finding Representative Fairness Metrics for a Given Context

It is of critical importance to be aware of the historical discriminatio...
research
11/18/2019

GPT Conjecture: Understanding the Trade-offs between Granularity, Performance and Timeliness in Control-Flow Integrity

Performance/security trade-off is widely noticed in CFI research, howeve...
research
10/07/2020

A Bandit-Based Algorithm for Fairness-Aware Hyperparameter Optimization

Considerable research effort has been guided towards algorithmic fairnes...

Please sign up or login with your details

Forgot password? Click here to reset