Improving Fairness in Criminal Justice Algorithmic Risk Assessments Using Optimal Transport and Conformal Prediction Sets

by   Richard A. Berk, et al.

In the United States and elsewhere, risk assessment algorithms are being used to help inform criminal justice decision-makers. A common intent is to forecast an offender's “future dangerousness.” Such algorithms have been correctly criticized for potential unfairness, and there is an active cottage industry trying to make repairs. In this paper, we use counterfactual reasoning to consider the prospects for improved fairness when members of a less privileged group are treated by a risk algorithm as if they are members of a more privileged group. We combine a machine learning classifier trained in a novel manner with an optimal transport adjustment for the relevant joint probability distributions, which together provide a constructive response to claims of bias-in-bias-out. A key distinction is between fairness claims that are empirically testable and fairness claims that are not. We then use confusion tables and conformal prediction sets to evaluate achieved fairness for projected risk. Our data are a random sample of 300,000 offenders at their arraignments for a large metropolitan area in the United States during which decisions to release or detain are made. We show that substantial improvement in fairness can be achieved consistent with a Pareto improvement for protected groups.


page 1

page 2

page 3

page 4


Improving Fairness in Criminal Justice Algorithmic Risk Assessments Using Conformal Prediction Sets

Risk assessment algorithms have been correctly criticized for potential ...

Nested Conformal Prediction Sets for Classification with Applications to Probation Data

Risk assessments to help inform criminal justice decisions have been use...

Everything is Relative: Understanding Fairness with Optimal Transport

To study discrimination in automated decision-making systems, scholars h...

Optimal Transport of Binary Classifiers to Fairness

Much of the past work on fairness in machine learning has focused on for...

Almost Politically Acceptable Criminal Justice Risk Assessment

In criminal justice risk forecasting, one can prove that it is impossibl...

Testing Group Fairness via Optimal Transport Projections

We present a statistical testing framework to detect if a given machine ...

The Fairness of Risk Scores Beyond Classification: Bipartite Ranking and the xAUC Metric

Where machine-learned predictive risk scores inform high-stakes decision...

Please sign up or login with your details

Forgot password? Click here to reset