DeepAI AI Chat
Log In Sign Up

Stable-Predictive Optimistic Counterfactual Regret Minimization

by   Gabriele Farina, et al.
Columbia University
Carnegie Mellon University

The CFR framework has been a powerful tool for solving large-scale extensive-form games in practice. However, the theoretical rate at which past CFR-based algorithms converge to the Nash equilibrium is on the order of O(T^-1/2), where T is the number of iterations. In contrast, first-order methods can be used to achieve a O(T^-1) dependence on iterations, yet these methods have been less successful in practice. In this work we present the first CFR variant that breaks the square-root dependence on iterations. By combining and extending recent advances on predictive and stable regret minimizers for the matrix-game setting we show that it is possible to leverage "optimistic" regret minimizers to achieve a O(T^-3/4) convergence rate within CFR. This is achieved by introducing a new notion of stable-predictivity, and by setting the stability of each counterfactual regret minimizer relative to its location in the decision tree. Experiments show that this method is faster than the original CFR algorithm, although not as fast as newer variants, in spite of their worst-case O(T^-1/2) dependence on iterations.


page 1

page 2

page 3

page 4


Regret Minimization in Behaviorally-Constrained Zero-Sum Games

No-regret learning has emerged as a powerful tool for solving extensive-...

Faster Game Solving via Predictive Blackwell Approachability: Connecting Regret Matching and Mirror Descent

Blackwell approachability is a framework for reasoning about repeated ga...

Optimistic Regret Minimization for Extensive-Form Games via Dilated Distance-Generating Functions

We study the performance of optimistic regret-minimization algorithms fo...

Theoretical and Practical Advances on Smoothing for Extensive-Form Games

Sparse iterative methods, in particular first-order methods, are known t...

Solving Imperfect-Information Games via Discounted Regret Minimization

Counterfactual regret minimization (CFR) is a family of iterative algori...

Faster Regret Matching

The regret matching algorithm proposed by Sergiu Hart is one of the most...

The Power of Regularization in Solving Extensive-Form Games

In this paper, we investigate the power of regularization, a common tech...