Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support

04/05/2022
by   Anna Kawakami, et al.
0

AI-based decision support tools (ADS) are increasingly used to augment human decision-making in high-stakes, social contexts. As public sector agencies begin to adopt ADS, it is critical that we understand workers' experiences with these systems in practice. In this paper, we present findings from a series of interviews and contextual inquiries at a child welfare agency, to understand how they currently make AI-assisted child maltreatment screening decisions. Overall, we observe how workers' reliance upon the ADS is guided by (1) their knowledge of rich, contextual information beyond what the AI model captures, (2) their beliefs about the ADS's capabilities and limitations relative to their own, (3) organizational pressures and incentives around the use of the ADS, and (4) awareness of misalignments between algorithmic predictions and their own decision-making objectives. Drawing upon these findings, we discuss design implications towards supporting more effective human-AI decision-making.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/30/2023

Training Towards Critical Use: Learning to Situate AI Predictions Relative to Human Knowledge

A growing body of research has explored how to support humans in making ...
research
04/12/2022

Heterogeneity in Algorithm-Assisted Decision-Making: A Case Study in Child Abuse Hotline Screening

Algorithmic risk assessment tools are now commonplace in public sector d...
research
06/07/2021

Algorithms and Decision-Making in the Public Sector

This article surveys the use of algorithmic systems to support decision-...
research
07/09/2023

Thriving Innovation Ecosystems: Synergy Among Stakeholders, Tools, and People

An innovation ecosystem is a multi-stakeholder environment, where differ...
research
03/06/2023

Stakeholder-Centered AI Design: Co-Designing Worker Tools with Gig Workers through Data Probes

AI technologies continue to advance from digital assistants to assisted ...
research
04/21/2019

Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes

Clinical decision support tools (DST) promise improved healthcare outcom...
research
04/29/2022

Extended Analysis of "How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions"

This is an extended analysis of our paper "How Child Welfare Workers Red...

Please sign up or login with your details

Forgot password? Click here to reset