DeepAI
Log In Sign Up

Online Double Oracle

03/13/2021
by   Le Cong Dinh, et al.
3

Solving strategic games with huge action space is a critical yet under-explored topic in economics, operations research and artificial intelligence. This paper proposes new learning algorithms for solving two-player zero-sum normal-form games where the number of pure strategies is prohibitively large. Specifically, we combine no-regret analysis from online learning with Double Oracle (DO) methods from game theory. Our method – Online Double Oracle (ODO) – is provably convergent to a Nash equilibrium (NE). Most importantly, unlike normal DO methods, ODO is rationale in the sense that each agent in ODO can exploit strategic adversary with a regret bound of 𝒪(√(T k log(k))) where k is not the total number of pure strategies, but rather the size of effective strategy set that is linearly dependent on the support size of the NE. On tens of different real-world games, ODO outperforms DO, PSRO methods, and no-regret algorithms such as Multiplicative Weight Update by a significant margin, both in terms of convergence rate to a NE and average payoff against strategic adversaries.

READ FULL TEXT

page 1

page 2

page 3

page 4

01/19/2022

Anytime PSRO for Two-Player Zero-Sum Games

Policy space response oracles (PSRO) is a multi-agent reinforcement lear...
03/11/2021

XDO: A Double Oracle Algorithm for Extensive-Form Games

Policy Space Response Oracles (PSRO) is a deep reinforcement learning al...
09/25/2020

Double Oracle Algorithm for Computing Equilibria in Continuous Games

Many efficient algorithms have been designed to recover Nash equilibria ...
10/05/2022

Game Theoretic Rating in N-player general-sum games with Equilibria

Rating strategies in a game is an important area of research in game the...
03/14/2021

Modelling Behavioural Diversity for Learning in Open-Ended Games

Promoting behavioural diversity is critical for solving games with non-t...
11/18/2021

Number of New Top 2

In this paper we compare the numbers of new top 2 USA annually since 198...
06/04/2021

Consensus Multiplicative Weights Update: Learning to Learn using Projector-based Game Signatures

Recently, Optimistic Multiplicative Weights Update (OMWU) was proven to ...