DeepAI AI Chat
Log In Sign Up

Last-iterate Convergence in Extensive-Form Games

06/27/2021
by   Chung-Wei Lee, et al.
0

Regret-based algorithms are highly efficient at finding approximate Nash equilibria in sequential games such as poker games. However, most regret-based algorithms, including counterfactual regret minimization (CFR) and its variants, rely on iterate averaging to achieve convergence. Inspired by recent advances on last-iterate convergence of optimistic algorithms in zero-sum normal-form games, we study this phenomenon in sequential games, and provide a comprehensive study of last-iterate convergence for zero-sum extensive-form games with perfect recall (EFGs), using various optimistic regret-minimization algorithms over treeplexes. This includes algorithms using the vanilla entropy or squared Euclidean norm regularizers, as well as their dilated versions which admit more efficient implementation. In contrast to CFR, we show that all of these algorithms enjoy last-iterate convergence, with some of them even converging exponentially fast. We also provide experiments to further support our theoretical results.

READ FULL TEXT

page 1

page 2

page 3

page 4

05/24/2023

Regret Matching+: (In)Stability and Fast Convergence in Games

Regret Matching+ (RM+) and its variants are important algorithms for sol...
04/11/2022

Equilibrium Finding in Normal-Form Games Via Greedy Regret Minimization

We extend the classic regret minimization framework for approximating eq...
04/23/2018

Analysis of Hannan Consistent Selection for Monte Carlo Tree Search in Simultaneous Move Games

Hannan consistency, or no external regret, is a key concept for learning...
04/24/2019

Solving zero-sum extensive-form games with arbitrary payoff uncertainty models

Modeling strategic conflict from a game theoretical perspective involves...
03/02/2023

Learning not to Regret

Regret minimization is a key component of many algorithms for finding Na...
10/07/2018

Solving Large Sequential Games with the Excessive Gap Technique

There has been tremendous recent progress on equilibrium-finding algorit...
10/07/2019

Combining No-regret and Q-learning

Counterfactual Regret Minimization (CFR) has found success in settings l...