Efficient Decentralized Learning Dynamics for Extensive-Form Coarse Correlated Equilibrium: No Expensive Computation of Stationary Distributions Required

09/16/2021
by   Gabriele Farina, et al.
0

While in two-player zero-sum games the Nash equilibrium is a well-established prescriptive notion of optimal play, its applicability as a prescriptive tool beyond that setting is limited. Consequently, the study of decentralized learning dynamics that guarantee convergence to correlated solution concepts in multiplayer, general-sum extensive-form (i.e., tree-form) games has become an important topic of active research. The per-iteration complexity of the currently known learning dynamics depends on the specific correlated solution concept considered. For example, in the case of extensive-form correlated equilibrium (EFCE), all known dynamics require, as an intermediate step at each iteration, to compute the stationary distribution of multiple Markov chains, an expensive operation in practice. Oppositely, in the case of normal-form coarse correlated equilibrium (NFCCE), simple no-external-regret learning dynamics that amount to a linear-time traversal of the tree-form decision space of each agent suffice to guarantee convergence. This paper focuses on extensive-form coarse correlated equilibrium (EFCCE), an intermediate solution concept that is a subset of NFCCE and a superset of EFCE. Being a superset of EFCE, any learning dynamics for EFCE automatically guarantees convergence to EFCCE. However, since EFCCE is a simpler solution concept, this begs the question: do learning dynamics for EFCCE that avoid the expensive computation of stationary distributions exist? This paper answers the previous question in the positive. Our learning dynamics only require the orchestration of no-external-regret minimizers, thus showing that EFCCE is more akin to NFCCE than to EFCE from a learning perspective. Our dynamics guarantees that the empirical frequency of play after T iteration is a O(1/√(T))-approximate EFCCE with high probability, and an EFCCE almost surely in the limit.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/01/2020

No-regret learning dynamics for extensive-form correlated and coarse correlated equilibria

Recently, there has been growing interest around less-restrictive soluti...
research
02/11/2022

Faster No-Regret Learning Dynamics for Extensive-Form Correlated and Coarse Correlated Equilibria

A recent emerging trend in the literature on learning in games has been ...
research
04/04/2021

Simple Uncoupled No-Regret Learning Dynamics for Extensive-Form Correlated Equilibrium

The existence of simple uncoupled no-regret learning dynamics that conve...
research
07/13/2023

Multi-Player Zero-Sum Markov Games with Networked Separable Interactions

We study a new class of Markov games (MGs), Multi-player Zero-sum Markov...
research
08/26/2019

Coarse Correlation in Extensive-Form Games

Coarse correlation models strategic interactions of rational agents comp...
research
12/29/2022

Safe Subgame Resolving for Extensive Form Correlated Equilibrium

Correlated Equilibrium is a solution concept that is more general than N...
research
04/23/2018

Analysis of Hannan Consistent Selection for Monte Carlo Tree Search in Simultaneous Move Games

Hannan consistency, or no external regret, is a key concept for learning...

Please sign up or login with your details

Forgot password? Click here to reset