Algorithmic risk assessments can alter human decision-making processes in high-stakes government contexts

by   Ben Green, et al.

Governments are increasingly turning to algorithmic risk assessments when making important decisions, believing that these algorithms will improve public servants' ability to make policy-relevant predictions and thereby lead to more informed decisions. Yet because many policy decisions require balancing risk-minimization with competing social goals, evaluating the impacts of risk assessments requires considering how public servants are influenced by risk assessments when making policy decisions rather than just how accurately these algorithms make predictions. Through an online experiment with 2,140 lay participants simulating two high-stakes government contexts, we provide the first large-scale evidence that risk assessments can systematically alter decision-making processes by increasing the salience of risk as a factor in decisions and that these shifts could exacerbate racial disparities. These results demonstrate that improving human prediction accuracy with algorithms does not necessarily improve human decisions and highlight the need to experimentally test how government algorithms are used by human decision-makers.



page 18

page 35


A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare

Algorithms have permeated throughout civil government and society, where...

The Flaws of Policies Requiring Human Oversight of Government Algorithms

Policymakers around the world are increasingly considering how to preven...

Compounding Injustice: History and Prediction in Carceral Decision-Making

Risk assessment algorithms in criminal justice put people's lives at the...

The Hidden Inconsistencies Introduced by Predictive Algorithms in Judicial Decision Making

Algorithms, from simple automation to machine learning, have been introd...

Catastrophe, Compounding Consistency in Choice

Conditional value-at-risk (CVaR) precisely characterizes the influence t...

Experimental Evaluation of Algorithm-Assisted Human Decision-Making: Application to Pretrial Public Safety Assessment

Despite an increasing reliance on fully-automated algorithmic decision m...

Protection or Peril of Following the Crowd in a Pandemic-Concurrent Flood Evacuation

The decisions of whether and how to evacuate during a climate disaster a...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.