Accounting for Model Uncertainty in Algorithmic Discrimination

05/10/2021
by   Junaid Ali, et al.
0

Traditional approaches to ensure group fairness in algorithmic decision making aim to equalize “total” error rates for different subgroups in the population. In contrast, we argue that the fairness approaches should instead focus only on equalizing errors arising due to model uncertainty (a.k.a epistemic uncertainty), caused due to lack of knowledge about the best model or due to lack of data. In other words, our proposal calls for ignoring the errors that occur due to uncertainty inherent in the data, i.e., aleatoric uncertainty. We draw a connection between predictive multiplicity and model uncertainty and argue that the techniques from predictive multiplicity could be used to identify errors made due to model uncertainty. We propose scalable convex proxies to come up with classifiers that exhibit predictive multiplicity and empirically show that our methods are comparable in performance and up to four orders of magnitude faster than the current state-of-the-art. We further propose methods to achieve our goal of equalizing group error rates arising due to model uncertainty in algorithmic decision making and demonstrate the effectiveness of these methods using synthetic and real-world datasets.

READ FULL TEXT
research
01/29/2023

Fair Decision-making Under Uncertainty

There has been concern within the artificial intelligence (AI) community...
research
09/28/2018

Active Fairness in Algorithmic Decision Making

Society increasingly relies on machine learning models for automated dec...
research
02/19/2021

Resolving the Disparate Impact of Uncertainty: Affirmative Action vs. Affirmative Information

Algorithmic risk assessments hold the promise of greatly advancing accur...
research
06/13/2018

Fairness Behind a Veil of Ignorance: A Welfare Analysis for Automated Decision Making

We draw attention to an important, yet largely overlooked aspect of eval...
research
04/04/2021

Pareto Efficient Fairness in Supervised Learning: From Extraction to Tracing

As algorithmic decision-making systems are becoming more pervasive, it i...
research
11/30/2022

Evidential Conditional Neural Processes

The Conditional Neural Process (CNP) family of models offer a promising ...
research
02/23/2022

Scalable Query Answering under Uncertainty to Neuroscientific Ontological Knowledge: The NeuroLang Approach

Researchers in neuroscience have a growing number of datasets available ...

Please sign up or login with your details

Forgot password? Click here to reset