CFR-MIX: Solving Imperfect Information Extensive-Form Games with Combinatorial Action Space

05/18/2021
by   Shuxin Li, et al.
0

In many real-world scenarios, a team of agents coordinate with each other to compete against an opponent. The challenge of solving this type of game is that the team's joint action space grows exponentially with the number of agents, which results in the inefficiency of the existing algorithms, e.g., Counterfactual Regret Minimization (CFR). To address this problem, we propose a new framework of CFR: CFR-MIX. Firstly, we propose a new strategy representation that represents a joint action strategy using individual strategies of all agents and a consistency relationship to maintain the cooperation between agents. To compute the equilibrium with individual strategies under the CFR framework, we transform the consistency relationship between strategies to the consistency relationship between the cumulative regret values. Furthermore, we propose a novel decomposition method over cumulative regret values to guarantee the consistency relationship between the cumulative regret values. Finally, we introduce our new algorithm CFR-MIX which employs a mixing layer to estimate cumulative regret values of joint actions as a non-linear combination of cumulative regret values of individual actions. Experimental results show that CFR-MIX outperforms existing algorithms on various games significantly.

READ FULL TEXT
research
12/27/2018

Double Neural Counterfactual Regret Minimization

Counterfactual Regret Minimization (CRF) is a fundamental and effective ...
research
09/06/2022

A Zeroth-Order Momentum Method for Risk-Averse Online Convex Games

We consider risk-averse learning in repeated unknown games where the goa...
research
09/12/2016

Reduced Space and Faster Convergence in Imperfect-Information Games via Regret-Based Pruning

Counterfactual Regret Minimization (CFR) is the most popular iterative a...
research
04/11/2022

A Unified Perspective on Deep Equilibrium Finding

Extensive-form games provide a versatile framework for modeling interact...
research
12/03/2020

Model-free Neural Counterfactual Regret Minimization with Bootstrap Learning

Counterfactual Regret Minimization (CFR) has achieved many fascinating r...
research
03/16/2022

Risk-Averse No-Regret Learning in Online Convex Games

We consider an online stochastic game with risk-averse agents whose goal...
research
12/06/2022

Regret Minimization with Dynamic Benchmarks in Repeated Games

In repeated games, strategies are often evaluated by their ability to gu...

Please sign up or login with your details

Forgot password? Click here to reset