Fairness Preferences, Actual and Hypothetical: A Study of Crowdworker Incentives

by   Angie Peng, et al.

How should we decide which fairness criteria or definitions to adopt in machine learning systems? To answer this question, we must study the fairness preferences of actual users of machine learning systems. Stringent parity constraints on treatment or impact can come with trade-offs, and may not even be preferred by the social groups in question (Zafar et al., 2017). Thus it might be beneficial to elicit what the group's preferences are, rather than rely on a priori defined mathematical fairness constraints. Simply asking for self-reported rankings of users is challenging because research has shown that there are often gaps between people's stated and actual preferences(Bernheim et al., 2013). This paper outlines a research program and experimental designs for investigating these questions. Participants in the experiments are invited to perform a set of tasks in exchange for a base payment–they are told upfront that they may receive a bonus later on, and the bonus could depend on some combination of output quantity and quality. The same group of workers then votes on a bonus payment structure, to elicit preferences. The voting is hypothetical (not tied to an outcome) for half the group and actual (tied to the actual payment outcome) for the other half, so that we can understand the relation between a group's actual preferences and hypothetical (stated) preferences. Connections and lessons from fairness in machine learning are explored.



There are no comments yet.


page 1

page 2

page 3

page 4


On Democratic Fairness for Groups of Agents

We study the problem of allocating indivisible goods to groups of intere...

50 Years of Test (Un)fairness: Lessons for Machine Learning

Quantitative definitions of what is unfair and what is fair have been in...

Group Fairness: Independence Revisited

This paper critically examines arguments against independence, a measure...

Distributional Individual Fairness in Clustering

In this paper, we initiate the study of fair clustering that ensures dis...

"Do the Right Thing" for Whom? An Experiment on Ingroup Favouritism, Group Assorting and Moral Suasion

In this paper we investigate the effect of moral suasion on ingroup favo...

An Empirical Study of Rich Subgroup Fairness for Machine Learning

Kearns et al. [2018] recently proposed a notion of rich subgroup fairnes...

Social versus Moral preferences in the Ultimatum Game: A theoretical model and an experiment

In the Ultimatum Game (UG) one player, named "proposer", has to decide h...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.