Social Diversity and Social Preferences in Mixed-Motive Reinforcement Learning

02/06/2020
by   Kevin R. McKee, et al.
4

Recent research on reinforcement learning in pure-conflict and pure-common interest games has emphasized the importance of population heterogeneity. In contrast, studies of reinforcement learning in mixed-motive games have primarily leveraged homogeneous approaches. Given the defining characteristic of mixed-motive games–the imperfect correlation of incentives between group members–we study the effect of population heterogeneity on mixed-motive reinforcement learning. We draw on interdependence theory from social psychology and imbue reinforcement learning agents with Social Value Orientation (SVO), a flexible formalization of preferences over group outcome distributions. We subsequently explore the effects of diversity in SVO on populations of reinforcement learning agents in two mixed-motive Markov games. We demonstrate that heterogeneity in SVO generates meaningful and complex behavioral variation among agents similar to that suggested by interdependence theory. Empirical results in these mixed-motive dilemmas suggest agents trained in heterogeneous populations develop particularly generalized, high-performing policies relative to those trained in homogeneous populations.

READ FULL TEXT

page 1

page 2

page 3

page 5

page 6

page 7

page 8

page 9

research
10/19/2021

Improved cooperation by balancing exploration and exploitation in intertemporal social dilemma tasks

When an individual's behavior has rational characteristics, this may lea...
research
05/01/2023

Heterogeneous Social Value Orientation Leads to Meaningful Diversity in Sequential Social Dilemmas

In social psychology, Social Value Orientation (SVO) describes an indivi...
research
05/19/2023

Learning Diverse Risk Preferences in Population-based Self-play

Among the great successes of Reinforcement Learning (RL), self-play algo...
research
11/09/2017

A Further Analysis of The Role of Heterogeneity in Coevolutionary Spatial Games

Heterogeneity has been studied as one of the most common explanations of...
research
05/03/2023

System Neural Diversity: Measuring Behavioral Heterogeneity in Multi-Agent Learning

Evolutionary science provides evidence that diversity confers resilience...
research
08/01/2018

Cooperative Group Optimization with Ants (CGO-AS): Leverage Optimization with Mixed Individual and Social Learning

We present CGO-AS, a generalized Ant System (AS) implemented in the fram...
research
04/27/2022

On the role of population heterogeneity in emergent communication

Populations have often been perceived as a structuring component for lan...

Please sign up or login with your details

Forgot password? Click here to reset