Simplicity Creates Inequity: Implications for Fairness, Stereotypes, and Interpretability

by   Jon Kleinberg, et al.

Algorithmic predictions are increasingly used to aid, or in some cases supplant, human decision-making, and this development has placed new demands on the outputs of machine learning procedures. To facilitate human interaction, we desire that they output prediction functions that are in some fashion simple or interpretable. And because they influence consequential decisions, we also desire equitable prediction functions, ones whose allocations benefit (or at the least do not harm) disadvantaged groups. We develop a formal model to explore the relationship between simplicity and equity. Although the two concepts appear to be motivated by qualitatively distinct goals, our main result shows a fundamental inconsistency between them. Specifically, we formalize a general framework for producing simple prediction functions, and in this framework we show that every simple prediction function is strictly improvable: there exists a more complex prediction function that is both strictly more efficient and also strictly more equitable. Put another way, using a simple prediction function both reduces utility for disadvantaged groups and reduces overall welfare. Our result is not only about algorithms but about any process that produces simple models, and as such connects to the psychology of stereotypes and to an earlier economics literature on statistical discrimination.


page 1

page 2

page 3

page 4


Towards a Fairness-Aware Scoring System for Algorithmic Decision-Making

Scoring systems, as simple classification models, have significant advan...

On the Fairness of Machine-Assisted Human Decisions

When machine-learning algorithms are deployed in high-stakes decisions, ...

Fairness for Whom? Critically reframing fairness with Nash Welfare Product

Recent studies on disparate impact in machine learning applications have...

A Justice-Based Framework for the Analysis of Algorithmic Fairness-Utility Trade-Offs

In prediction-based decision-making systems, different perspectives can ...

Distributive Justice and Fairness Metrics in Automated Decision-making: How Much Overlap Is There?

The advent of powerful prediction algorithms led to increased automation...

Fairer and more accurate, but for whom?

Complex statistical machine learning models are increasingly being used ...

Please sign up or login with your details

Forgot password? Click here to reset