DeepAI AI Chat
Log In Sign Up

Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support

12/10/2021
by   Michael Madaio, et al.
0

Various tools and practices have been developed to support practitioners in identifying, assessing, and mitigating fairness-related harms caused by AI systems. However, prior research has highlighted gaps between the intended design of these tools and practices and their use within particular contexts, including gaps caused by the role that organizational factors play in shaping fairness work. In this paper, we investigate these gaps for one such practice: disaggregated evaluations of AI systems, intended to uncover performance disparities between demographic groups. By conducting semi-structured interviews and structured workshops with thirty-three AI practitioners from ten teams at three technology companies, we identify practitioners' processes, challenges, and needs for support when designing disaggregated evaluations. We find that practitioners face challenges when choosing performance metrics, identifying the most relevant direct stakeholders and demographic groups on which to focus, and collecting datasets with which to conduct disaggregated evaluations. More generally, we identify impacts on fairness work stemming from a lack of engagement with direct stakeholders, business imperatives that prioritize customers over marginalized groups, and the drive to deploy AI systems at scale.

READ FULL TEXT

page 1

page 2

page 3

page 4

03/29/2022

Exploring Opportunities in Usable Hazard Analysis Processes for AI Engineering

Embedding artificial intelligence into systems introduces significant ch...
09/26/2019

This Thing Called Fairness: Disciplinary Confusion Realizing a Value in Technology

The explosion in the use of software in important sociotechnical systems...
03/10/2021

Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs

Several pieces of work have uncovered performance disparities by conduct...
03/29/2023

Fairlearn: Assessing and Improving Fairness of AI Systems

Fairlearn is an open source project to help practitioners assess and imp...
01/23/2023

The Reasonable Effectiveness of Diverse Evaluation Data

In this paper, we present findings from an semi-experimental exploration...
02/20/2020

The Problem with Metrics is a Fundamental Problem for AI

Optimizing a given metric is a central aspect of most current AI approac...