Adaptive Learning in Continuous Games: Optimal Regret Bounds and Convergence to Nash Equilibrium

by   Yu-Guan Hsieh, et al.

In game-theoretic learning, several agents are simultaneously following their individual interests, so the environment is non-stationary from each player's perspective. In this context, the performance of a learning algorithm is often measured by its regret. However, no-regret algorithms are not created equal in terms of game-theoretic guarantees: depending on how they are tuned, some of them may drive the system to an equilibrium, while others could produce cyclic, chaotic, or otherwise divergent trajectories. To account for this, we propose a range of no-regret policies based on optimistic mirror descent, with the following desirable properties: i) they do not require any prior tuning or knowledge of the game; ii) they all achieve O(√(T)) regret against arbitrary, adversarial opponents; and iii) they converge to the best response against convergent opponents. Also, if employed by all players, then iv) they guarantee O(1) social regret; while v) the induced sequence of play converges to Nash equilibrium with O(1) individual regret in all variationally stable games (a class of games that includes all monotone and convex-concave zero-sum games).



There are no comments yet.


page 1

page 2

page 3

page 4


Successful Nash Equilibrium Agent for a 3-Player Imperfect-Information Game

Creating strong agents for games with more than two players is a major o...

Learning in time-varying games

In this paper, we examine the long-term behavior of regret-minimizing ag...

Acceleration through Optimistic No-Regret Dynamics

We consider the problem of minimizing a smooth convex function by reduci...

Let's be honest: An optimal no-regret framework for zero-sum games

We revisit the problem of solving two-player zero-sum games in the decen...

Online Monotone Games

Algorithmic game theory (AGT) focuses on the design and analysis of algo...

Gradient-free Online Learning in Games with Delayed Rewards

Motivated by applications to online advertising and recommender systems,...

Survival of the strictest: Stable and unstable equilibria under regularized learning with partial information

In this paper, we examine the Nash equilibrium convergence properties of...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.