Lazy-CFR: a fast regret minimization algorithm for extensive games with imperfect information
share
In this paper, we focus on solving two-player zero-sum extensive games with imperfect information. Counterfactual regret minimization (CFR) is the most popular algorithm on solving such games and achieves state-of-the-art performance in practice. However, the performance of CFR is not fully understood, since empirical results on the regret are much better than the upper bound proved in zinkevich2008regret. Another issue of CFR is that CFR has to traverse the whole game tree in each round, which is not tolerable in large scale games. In this paper, we present a novel technique, lazy update, which can avoid traversing the whole game tree in CFR. Further, we present a novel analysis on the CFR with lazy update. Our analysis can also be applied to the vanilla CFR, which results in a much tighter regret bound than that proved in zinkevich2008regret. Inspired by lazy update, we further present a novel CFR variant, named Lazy-CFR. Compared to traversing O(|I|) information sets in vanilla CFR, Lazy-CFR needs only to traverse O(√(|I|)) information sets per round while the regret bound almost keep the same, where I is the class of all information sets. As a result, Lazy-CFR shows better convergence result compared with vanilla CFR. Experimental results consistently show that Lazy-CFR outperforms the vanilla CFR significantly.
READ FULL TEXT