Decision-makers Processing of AI Algorithmic Advice: Automation Bias versus Selective Adherence

03/03/2021
by   Saar Alon-Barkat, et al.
0

Artificial intelligence algorithms are increasingly adopted as decisional aides by public organisations, with the promise of overcoming biases of human decision-makers. At the same time, the use of algorithms may introduce new biases in the human-algorithm interaction. A key concern emerging from psychology studies regards human overreliance on algorithmic advice even in the face of warning signals and contradictory information from other sources (automation bias). A second concern regards decision-makers inclination to selectively adopt algorithmic advice when it matches their pre-existing beliefs and stereotypes (selective adherence). To date, we lack rigorous empirical evidence about the prevalence of these biases in a public sector context. We assess these via two pre-registered experimental studies (N=1,509), simulating the use of algorithmic advice in decisions pertaining to the employment of school teachers in the Netherlands. In study 1, we test automation bias by exploring participants adherence to a prediction of teachers performance, which contradicts additional evidence, while comparing between two types of predictions: algorithmic v. human-expert. We do not find evidence for automation bias. In study 2, we replicate these findings, and we also test selective adherence by manipulating the teachers ethnic background. We find a propensity for adherence when the advice predicts low performance for a teacher of a negatively stereotyped ethnic minority, with no significant differences between algorithmic and human advice. Overall, our findings of selective, biased adherence belie the promise of neutrality that has propelled algorithm use in the public sector.

READ FULL TEXT
research
05/03/2021

Algorithms are not neutral: Bias in collaborative filtering

Discussions of algorithmic bias tend to focus on examples where either t...
research
03/28/2019

The Algorithmic Automation Problem: Prediction, Triage, and Human Effort

In a wide array of areas, algorithms are matching and surpassing the per...
research
02/21/2018

Algorithmic Collusion in Cournot Duopoly Market: Evidence from Experimental Economics

Algorithmic collusion is an emerging concept in current artificial intel...
research
06/24/2021

"Part Man, Part Machine, All Cop": Automation in Policing

Digitisation, automation and datafication permeate policing and justice ...
research
06/02/2023

Auditing for Human Expertise

High-stakes prediction tasks (e.g., patient diagnosis) are often handled...
research
10/18/2017

Exploiting oddsmaker bias to improve the prediction of NFL outcomes

Accurately predicting the outcome of sporting events has been a goal for...
research
06/28/2023

Bending the Automation Bias Curve: A Study of Human and AI-based Decision Making in National Security Contexts

Uses of artificial intelligence (AI), especially those powered by machin...

Please sign up or login with your details

Forgot password? Click here to reset