Documenting Data Production Processes: A Participatory Approach for Data Work

07/11/2022
by   Milagros Miceli, et al.
0

The opacity of machine learning data is a significant threat to ethical data work and intelligible systems. Previous research has addressed this issue by proposing standardized checklists to document datasets. This paper expands that field of inquiry by proposing a shift of perspective: from documenting datasets toward documenting data production. We draw on participatory design and collaborate with data workers at two companies located in Bulgaria and Argentina, where the collection and annotation of data for machine learning are outsourced. Our investigation comprises 2.5 years of research, including 33 semi-structured interviews, five co-design workshops, the development of prototypes, and several feedback instances with participants. We identify key challenges and requirements related to the integration of documentation practices in real-world data production scenarios. Our findings comprise important design considerations and highlight the value of designing data documentation based on the needs of data workers. We argue that a view of documentation as a boundary object, i.e., an object that can be used differently across organizations and teams but holds enough immutable content to maintain integrity, can be useful when designing documentation to retrieve heterogeneous, often distributed, contexts of data production.

READ FULL TEXT

page 7

page 8

page 10

page 17

page 29

page 31

page 32

page 33

research
05/24/2022

The Data-Production Dispositif

Machine learning (ML) depends on data to train and verify models. Very o...
research
09/16/2021

Studying Up Machine Learning Data: Why Talk About Bias When We Mean Power?

Research in machine learning (ML) has primarily argued that models train...
research
03/13/2020

Designing Tools for Semi-Automated Detection of Machine Learning Biases: An Interview Study

Machine learning models often make predictions that bias against certain...
research
03/06/2023

Stakeholder-Centered AI Design: Co-Designing Worker Tools with Gig Workers through Data Probes

AI technologies continue to advance from digital assistants to assisted ...
research
09/16/2022

Operationalizing Machine Learning: An Interview Study

Organizations rely on machine learning engineers (MLEs) to operationaliz...
research
05/16/2023

Walking the Walk of AI Ethics: Organizational Challenges and the Individualization of Risk among Ethics Entrepreneurs

Amidst decline in public trust in technology, computing ethics have take...
research
12/04/2022

Tragic Talkers: A Shakespearean Sound- and Light-Field Dataset for Audio-Visual Machine Learning Research

3D audio-visual production aims to deliver immersive and interactive exp...

Please sign up or login with your details

Forgot password? Click here to reset