Scaling up Psychology via Scientific Regret Minimization: A Case Study in Moral Decision-Making

10/16/2019
by   Mayank Agrawal, et al.
23

Do large datasets provide value to psychologists? Without a systematic methodology for working with such datasets, there is a valid concern that analyses will produce noise artifacts rather than true effects. In this paper, we offer a way to enable researchers to systematically build models and identify novel phenomena in large datasets. One traditional approach is to analyze the residuals of models—the biggest errors they make in predicting the data—to discover what might be missing from those models. However, once a dataset is sufficiently large, machine learning algorithms approximate the true underlying function better than the data, suggesting instead that the predictions of these data-driven models should be used to guide model-building. We call this approach "Scientific Regret Minimization" (SRM) as it focuses on minimizing errors for cases that we know should have been predictable. We demonstrate this methodology on a subset of the Moral Machine dataset, a public collection of roughly forty million moral decisions. Using SRM, we found that incorporating a set of deontological principles that capture dimensions along which groups of agents can vary (e.g. sex and age) improves a computational model of human moral judgment. Furthermore, we were able to identify and independently validate three interesting moral phenomena: criminal dehumanization, age of responsibility, and asymmetric notions of responsibility.

READ FULL TEXT

page 4

page 19

page 35

page 37

page 39

page 40

page 41

page 42

research
02/18/2019

Using Machine Learning to Guide Cognitive Modeling: A Case Study in Moral Reasoning

Large-scale behavioral datasets enable researchers to use complex machin...
research
10/03/2019

Bounds for Approximate Regret-Matching Algorithms

A dominant approach to solving large imperfect-information games is Coun...
research
06/20/2023

Statistical Tests for Replacing Human Decision Makers with Algorithms

This paper proposes a statistical framework with which artificial intell...
research
10/27/2022

Regret Bounds and Experimental Design for Estimate-then-Optimize

In practical applications, data is used to make decisions in two steps: ...
research
12/22/2021

Algorithmic Probability of Large Datasets and the Simplicity Bubble Problem in Machine Learning

When mining large datasets in order to predict new data, limitations of ...
research
09/21/2022

The Objective Function: Science and Society in the Age of Machine Intelligence

Machine intelligence, or the use of complex computational and statistica...
research
12/06/2020

Social Responsibility of Algorithms

Should we be concerned by the massive use of devices and algorithms whic...

Please sign up or login with your details

Forgot password? Click here to reset