Malthusian Reinforcement Learning

12/17/2018
by   Joel Z. Leibo, et al.
10

Here we explore a new algorithmic framework for multi-agent reinforcement learning, called Malthusian reinforcement learning, which extends self-play to include fitness-linked population size dynamics that drive ongoing innovation. In Malthusian RL, increases in a subpopulation's average return drive subsequent increases in its size, just as Thomas Malthus argued in 1798 was the relationship between preindustrial income levels and population growth. Malthusian reinforcement learning harnesses the competitive pressures arising from growing and shrinking population size to drive agents to explore regions of state and policy spaces that they could not otherwise reach. Furthermore, in environments where there are potential gains from specialization and division of labor, we show that Malthusian reinforcement learning is better positioned to take advantage of such synergies than algorithms based on self-play.

READ FULL TEXT

page 6

page 7

research
02/15/2018

Mean Field Multi-Agent Reinforcement Learning

Existing multi-agent reinforcement learning methods are limited typicall...
research
02/16/2021

Quantifying environment and population diversity in multi-agent reinforcement learning

Generalization is a major challenge for multi-agent reinforcement learni...
research
07/06/2021

Survey of Self-Play in Reinforcement Learning

In reinforcement learning (RL), the term self-play describes a kind of m...
research
05/19/2023

Learning Diverse Risk Preferences in Population-based Self-play

Among the great successes of Reinforcement Learning (RL), self-play algo...
research
07/13/2022

Self-Play PSRO: Toward Optimal Populations in Two-Player Zero-Sum Games

In competitive two-agent environments, deep reinforcement learning (RL) ...
research
09/03/2020

Optimality-based Analysis of XCSF Compaction in Discrete Reinforcement Learning

Learning classifier systems (LCSs) are population-based predictive syste...
research
03/02/2023

Population-based Evaluation in Repeated Rock-Paper-Scissors as a Benchmark for Multiagent Reinforcement Learning

Progress in fields of machine learning and adversarial planning has bene...

Please sign up or login with your details

Forgot password? Click here to reset