Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Pot

07/14/2021
by   Joel Z. Leibo, et al.
9

Existing evaluation suites for multi-agent reinforcement learning (MARL) do not assess generalization to novel situations as their primary objective (unlike supervised-learning benchmarks). Our contribution, Melting Pot, is a MARL evaluation suite that fills this gap, and uses reinforcement learning to reduce the human labor required to create novel test scenarios. This works because one agent's behavior constitutes (part of) another agent's environment. To demonstrate scalability, we have created over 80 unique test scenarios covering a broad range of research topics such as social dilemmas, reciprocity, resource sharing, and task partitioning. We apply these test scenarios to standard MARL training algorithms, and demonstrate how Melting Pot reveals weaknesses not apparent from training performance alone.

READ FULL TEXT

page 5

page 8

page 9

page 10

08/24/2019

Universal Policies to Learn Them All

We explore a collaborative and cooperative multi-agent reinforcement lea...
02/27/2021

Multi-agent Reinforcement Learning in OpenSpiel: A Reproduction Report

In this report, we present results reproductions for several core algori...
06/20/2022

From Multi-agent to Multi-robot: A Scalable Training and Evaluation Platform for Multi-robot Reinforcement Learning

Multi-agent reinforcement learning (MARL) has been gaining extensive att...
05/25/2021

KnowSR: Knowledge Sharing among Homogeneous Agents in Multi-agent Reinforcement Learning

Recently, deep reinforcement learning (RL) algorithms have made great pr...
02/15/2021

Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharing

Sharing parameters in multi-agent deep reinforcement learning has played...
01/07/2021

CoachNet: An Adversarial Sampling Approach for Reinforcement Learning

Despite the recent successes of reinforcement learning in games and robo...
10/29/2019

Generalization of Reinforcement Learners with Working and Episodic Memory

Memory is an important aspect of intelligence and plays a role in many d...