Convergence of Learning Dynamics in Stackelberg Games

by   Tanner Fiez, et al.
University of Washington

This paper investigates the convergence of learning dynamics in Stackelberg games. In the class of games we consider, there is a hierarchical game being played between a leader and a follower with continuous action spaces. We show that in zero-sum games, the only stable attractors of the Stackelberg gradient dynamics are Stackelberg equilibria. This insight allows us to develop a gradient-based update for the leader that converges to Stackelberg equilibria in zero-sum games and the set of stable attractors in general-sum games. We then consider a follower employing a gradient-play update rule instead of a best response strategy and propose a two-timescale algorithm with similar asymptotic convergence results. For this algorithm, we also provide finite-time high probability bounds for local convergence to a neighborhood of a stable Stackelberg equilibrium in general-sum games.


page 1

page 2

page 3

page 4


Stability of Gradient Learning Dynamics in Continuous Games: Vector Action Spaces

Towards characterizing the optimization landscape of games, this paper a...

Learning in nonatomic games, Part I: Finite action spaces and population games

We examine the long-run behavior of a wide range of dynamics for learnin...

A regularized variance-reduced modified extragradient method for stochastic hierarchical games

The theory of learning in games has so far focused mainly on games with ...

Solving Structured Hierarchical Games Using Differential Backward Induction

Many real-world systems possess a hierarchical structure where a strateg...

A Slingshot Approach to Learning in Monotone Games

In this paper, we address the problem of computing equilibria in monoton...

Safe Search for Stackelberg Equilibria in Extensive-Form Games

Stackelberg equilibrium is a solution concept in two-player games where ...

Robust Commitments and Partial Reputation

Agents rarely act in isolation -- their behavioral history, in particula...

Please sign up or login with your details

Forgot password? Click here to reset