Feedback Effects in Repeat-Use Criminal Risk Assessments

by   Benjamin Laufer, et al.

In the criminal legal context, risk assessment algorithms are touted as data-driven, well-tested tools. Studies known as validation tests are typically cited by practitioners to show that a particular risk assessment algorithm has predictive accuracy, establishes legitimate differences between risk groups, and maintains some measure of group fairness in treatment. To establish these important goals, most tests use a one-shot, single-point measurement. Using a Polya Urn model, we explore the implication of feedback effects in sequential scoring-decision processes. We show through simulation that risk can propagate over sequential decisions in ways that are not captured by one-shot tests. For example, even a very small or undetectable level of bias in risk allocation can amplify over sequential risk-based decisions, leading to observable group differences after a number of decision iterations. Risk assessment tools operate in a highly complex and path-dependent process, fraught with historical inequity. We conclude from this study that these tools do not properly account for compounding effects, and require new approaches to development and auditing.



page 1

page 2

page 3

page 4


Compounding Injustice: History and Prediction in Carceral Decision-Making

Risk assessment algorithms in criminal justice put people's lives at the...

Heterogeneity in Algorithm-Assisted Decision-Making: A Case Study in Child Abuse Hotline Screening

Algorithmic risk assessment tools are now commonplace in public sector d...

Fairness Evaluation in Presence of Biased Noisy Labels

Risk assessment tools are widely used around the country to inform decis...

Improving Fairness in Criminal Justice Algorithmic Risk Assessments Using Conformal Prediction Sets

Risk assessment algorithms have been correctly criticized for potential ...

Fairness with Dynamics

It has recently been shown that if feedback effects of decisions are ign...

Towards Simulation and Risk Assessment of Weather-Related Cascading Outages

Weather and environmental factors are verified to have played significan...

Almost Politically Acceptable Criminal Justice Risk Assessment

In criminal justice risk forecasting, one can prove that it is impossibl...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.