Efficient Regret Minimization Algorithm for Extensive-Form Correlated Equilibrium

10/28/2019
by   Gabriele Farina, et al.
0

Self-play methods based on regret minimization have become the state of the art for computing Nash equilibria in large two-players zero-sum extensive-form games. These methods fundamentally rely on the hierarchical structure of the players' sequential strategy spaces to construct a regret minimizer that recursively minimizes regret at each decision point in the game tree. In this paper, we introduce the first efficient regret minimization algorithm for computing extensive-form correlated equilibria in large two-player general-sum games with no chance moves. Designing such an algorithm is significantly more challenging than designing one for the Nash equilibrium counterpart, as the constraints that define the space of correlation plans lack the hierarchical structure and might even form cycles. We show that some of the constraints are redundant and can be excluded from consideration, and present an efficient algorithm that generates the space of extensive-form correlation plans incrementally from the remaining constraints. This structural decomposition is achieved via a special convexity-preserving operation that we coin scaled extension. We show that a regret minimizer can be designed for a scaled extension of any two convex sets, and that from the decomposition we then obtain a global regret minimizer. Our algorithm produces feasible iterates. Experiments show that it significantly outperforms prior approaches and for larger problems it is the only viable option.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/01/2020

No-regret learning dynamics for extensive-form correlated and coarse correlated equilibria

Recently, there has been growing interest around less-restrictive soluti...
research
12/29/2022

Safe Subgame Resolving for Extensive Form Correlated Equilibrium

Correlated Equilibrium is a solution concept that is more general than N...
research
05/27/2021

Better Regularization for Sequential Decision Spaces: Fast Convergence Rates for Nash, Correlated, and Team Equilibria

We study the application of iterative first-order methods to the problem...
research
11/06/2018

Composability of Regret Minimizers

Regret minimization is a powerful tool for solving large-scale problems;...
research
11/06/2018

Regret Circuits: Composability of Regret Minimizers

Regret minimization is a powerful tool for solving large-scale problems;...
research
10/18/2022

Exploitability Minimization in Games and Beyond

Pseudo-games are a natural and well-known generalization of normal-form ...
research
02/13/2021

Efficient Deviation Types and Learning for Hindsight Rationality in Extensive-Form Games

Hindsight rationality is an approach to playing multi-agent, general-sum...

Please sign up or login with your details

Forgot password? Click here to reset