Robust No-Regret Learning in Min-Max Stackelberg Games

by   Denizalp Goktas, et al.

The behavior of no-regret learning algorithms is well understood in two-player min-max (i.e, zero-sum) games. In this paper, we investigate the behavior of no-regret learning in min-max games with dependent strategy sets, where the strategy of the first player constrains the behavior of the second. Such games are best understood as sequential, i.e., min-max Stackelberg, games. We consider two settings, one in which only the first player chooses their actions using a no-regret algorithm while the second player best responds, and one in which both players use no-regret algorithms. For the former case, we show that no-regret dynamics converge to a Stackelberg equilibrium. For the latter case, we introduce a new type of regret, which we call Lagrangian regret, and show that if both players minimize their Lagrangian regrets, then play converges to a Stackelberg equilibrium. We then observe that online mirror descent (OMD) dynamics in these two settings correspond respectively to a known nested (i.e., sequential) gradient descent-ascent (GDA) algorithm and a new simultaneous GDA-like algorithm, thereby establishing convergence of these algorithms to Stackelberg equilibrium. Finally, we analyze the robustness of OMD dynamics to perturbations by investigating online min-max Stackelberg games. We prove that OMD dynamics are robust for a large class of online min-max games with independent strategy sets. In the dependent case, we demonstrate the robustness of OMD dynamics experimentally by simulating them in online Fisher markets, a canonical example of a min-max Stackelberg game with dependent strategy sets.



page 1

page 2

page 3

page 4


On Last-Iterate Convergence Beyond Zero-Sum Games

Most existing results about last-iterate convergence of learning dynamic...

Convex-Concave Min-Max Stackelberg Games

Min-max optimization problems (i.e., min-max games) have been attracting...

Alternating Mirror Descent for Constrained Min-Max Games

In this paper we study two-player bilinear zero-sum games with constrain...

LEAD: Least-Action Dynamics for Min-Max Optimization

Adversarial formulations such as generative adversarial networks (GANs) ...

No-Regret Dynamics in the Fenchel Game: A Unified Framework for Algorithmic Convex Optimization

We develop an algorithmic framework for solving convex optimization prob...

How to correctly prune tropical trees

We present tropical games, a generalization of combinatorial min-max gam...

A Provably Convergent and Practical Algorithm for Min-max Optimization with Applications to GANs

We present a new algorithm for optimizing min-max loss functions that ar...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.