Closing the AI Knowledge Gap

by   Ziv Epstein, et al.

AI researchers employ not only the scientific method, but also methodology from mathematics and engineering. However, the use of the scientific method - specifically hypothesis testing - in AI is typically conducted in service of engineering objectives. Growing interest in topics such as fairness and algorithmic bias show that engineering-focused questions only comprise a subset of the important questions about AI systems. This results in the AI Knowledge Gap: the number of unique AI systems grows faster than the number of studies that characterize these systems' behavior. To close this gap, we argue that the study of AI could benefit from the greater inclusion of researchers who are well positioned to formulate and test hypotheses about the behavior of AI systems. We examine the barriers preventing social and behavioral scientists from conducting such studies. Our diagnosis suggests that accelerating the scientific study of AI systems requires new incentives for academia and industry, mediated by new tools and institutions. To address these needs, we propose a two-sided marketplace called TuringBox. On one side, AI contributors upload existing and novel algorithms to be studied scientifically by others. On the other side, AI examiners develop and post machine intelligence tasks designed to evaluate and characterize algorithmic behavior. We discuss this market's potential to democratize the scientific study of AI behavior, and thus narrow the AI Knowledge Gap.


page 1

page 2

page 3

page 4


The Managerial Effects of Algorithmic Fairness Activism

How do ethical arguments affect AI adoption in business? We randomly exp...

Towards Language-driven Scientific AI

Inspired by recent and revolutionary developments in AI, particularly in...

Learning from learning machines: a new generation of AI technology to meet the needs of science

We outline emerging opportunities and challenges to enhance the utility ...

What Makes a Popular Academic AI Repository?

Many AI researchers are publishing code, data and other resources that a...

AI and Blackness: Towards moving beyond bias and representation

In this paper, we argue that AI ethics must move beyond the concepts of ...

FATE in AI: Towards Algorithmic Inclusivity and Accessibility

One of the defining phenomena in this age is the widespread deployment o...

External Stability Auditing to Test the Validity of Personality Prediction in AI Hiring

Automated hiring systems are among the fastest-developing of all high-st...

Please sign up or login with your details

Forgot password? Click here to reset