DeepAI AI Chat
Log In Sign Up

Data and its (dis)contents: A survey of dataset development and use in machine learning research

by   Amandalynne Paullada, et al.

Datasets have played a foundational role in the advancement of machine learning research. They form the basis for the models we design and deploy, as well as our primary medium for benchmarking and evaluation. Furthermore, the ways in which we collect, construct and share these datasets inform the kinds of problems the field pursues and the methods explored in algorithm development. However, recent work from a breadth of perspectives has revealed the limitations of predominant practices in dataset collection and use. In this paper, we survey the many concerns raised about the way we collect and use data in machine learning and advocate that a more cautious and thorough understanding of data is necessary to address several of the practical and ethical issues of the field.


page 1

page 2

page 3

page 4


Bringing the People Back In: Contesting Benchmark Machine Learning Datasets

In response to algorithmic unfairness embedded in sociotechnical systems...

Reduced, Reused and Recycled: The Life of a Dataset in Machine Learning Research

Benchmark datasets play a central role in the organization of machine le...

Challenges in Deploying Machine Learning: a Survey of Case Studies

In recent years, machine learning has received increased interest both a...

Machine Learning-based EEG Applications and Markets

This paper addresses both the various EEG applications and the current E...

HARK Side of Deep Learning -- From Grad Student Descent to Automated Machine Learning

Recent advancements in machine learning research, i.e., deep learning, i...

A Reflection on Learning from Data: Epistemology Issues and Limitations

Although learning from data is effective and has achieved significant mi...

Documenting Data Production Processes: A Participatory Approach for Data Work

The opacity of machine learning data is a significant threat to ethical ...