Multi-Agent Training beyond Zero-Sum with Correlated Equilibrium Meta-Solvers

06/17/2021
by   Luke Marris, et al.
0

Two-player, constant-sum games are well studied in the literature, but there has been limited progress outside of this setting. We propose Joint Policy-Space Response Oracles (JPSRO), an algorithm for training agents in n-player, general-sum extensive form games, which provably converges to an equilibrium. We further suggest correlated equilibria (CE) as promising meta-solvers, and propose a novel solution concept Maximum Gini Correlated Equilibrium (MGCE), a principled and computationally efficient family of solutions for solving the correlated equilibrium selection problem. We conduct several experiments using CE meta-solvers for JPSRO and demonstrate convergence on n-player, general-sum games.

READ FULL TEXT

page 1

page 2

page 3

page 4

04/01/2020

No-Regret Learning Dynamics for Extensive-Form Correlated Equilibrium

The existence of simple, uncoupled no-regret dynamics that converge to c...
12/10/2020

Hindsight and Sequential Rationality of Correlated Play

Driven by recent successes in two-player, zero-sum game solving and play...
09/28/2022

Meta-Learning in Games

In the literature on game-theoretic equilibrium finding, focus has mainl...
09/27/2019

A Generalized Training Approach for Multiagent Learning

This paper investigates a population-based training regime based on game...
11/11/2017

Practical Scalability for Stackelberg Security Games

Stackelberg Security Games (SSGs) have been adopted widely for modeling ...
06/24/2021

Exploration-Exploitation in Multi-Agent Competition: Convergence with Bounded Rationality

The interplay between exploration and exploitation in competitive multi-...
06/05/2020

Sparsified Linear Programming for Zero-Sum Equilibrium Finding

Computational equilibrium finding in large zero-sum extensive-form imper...