Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learning

by   Eun Seo Jo, et al.

A growing body of work shows that many problems in fairness, accountability, transparency, and ethics in machine learning systems are rooted in decisions surrounding the data collection and annotation process. In spite of its fundamental nature however, data collection remains an overlooked part of the machine learning (ML) pipeline. In this paper, we argue that a new specialization should be formed within ML that is focused on methodologies for data collection and annotation: efforts that require institutional frameworks and procedures. Specifically for sociocultural data, parallels can be drawn from archives and libraries. Archives are the longest standing communal effort to gather human information and archive scholars have already developed the language and procedures to address and discuss many challenges pertaining to data collection such as consent, power, inclusivity, transparency, and ethics privacy. We discuss these five key approaches in document collection practices in archives that can inform data collection in sociocultural ML. By showing data collection practices from another field, we encourage ML research to be more cognizant and systematic in data collection and draw from interdisciplinary expertise.


page 2

page 3

page 5

page 7

page 8

page 9

page 10

page 11


Designing Data: Proactive Data Collection and Iteration for Machine Learning

Lack of diversity in data collection has caused significant failures in ...

In Search of netUnicorn: A Data-Collection Platform to Develop Generalizable ML Models for Network Security Problems

The remarkable success of the use of machine learning-based solutions fo...

Easy, Reproducible and Quality-Controlled Data Collection with Crowdaq

High-quality and large-scale data are key to success for AI systems. How...

Towards Formalizing HRI Data Collection Processes

Within the human-robot interaction (HRI) community, many researchers hav...

Prespecification of Structure for Optimizing Data Collection and Research Transparency by Leveraging Conditional Independencies

Data collection and research methodology represents a critical part of t...

Detecting discriminatory risk through data annotation based on Bayesian inferences

Thanks to the increasing growth of computational power and data availabi...

Collect, Measure, Repeat: Reliability Factors for Responsible AI Data Collection

The rapid entry of machine learning approaches in our daily activities a...

Please sign up or login with your details

Forgot password? Click here to reset