Convergence of Learning Dynamics in Stackelberg Games

06/04/2019
by   Tanner Fiez, et al.
University of Washington
0

This paper investigates the convergence of learning dynamics in Stackelberg games. In the class of games we consider, there is a hierarchical game being played between a leader and a follower with continuous action spaces. We show that in zero-sum games, the only stable attractors of the Stackelberg gradient dynamics are Stackelberg equilibria. This insight allows us to develop a gradient-based update for the leader that converges to Stackelberg equilibria in zero-sum games and the set of stable attractors in general-sum games. We then consider a follower employing a gradient-play update rule instead of a best response strategy and propose a two-timescale algorithm with similar asymptotic convergence results. For this algorithm, we also provide finite-time high probability bounds for local convergence to a neighborhood of a stable Stackelberg equilibrium in general-sum games.

READ FULL TEXT

page 1

page 2

page 3

page 4

11/07/2020

Stability of Gradient Learning Dynamics in Continuous Games: Vector Action Spaces

Towards characterizing the optimization landscape of games, this paper a...
07/04/2021

Learning in nonatomic games, Part I: Finite action spaces and population games

We examine the long-run behavior of a wide range of dynamics for learnin...
02/13/2023

A regularized variance-reduced modified extragradient method for stochastic hierarchical games

The theory of learning in games has so far focused mainly on games with ...
06/08/2021

Solving Structured Hierarchical Games Using Differential Backward Induction

Many real-world systems possess a hierarchical structure where a strateg...
05/26/2023

A Slingshot Approach to Learning in Monotone Games

In this paper, we address the problem of computing equilibria in monoton...
02/02/2021

Safe Search for Stackelberg Equilibria in Extensive-Form Games

Stackelberg equilibrium is a solution concept in two-player games where ...
05/28/2019

Robust Commitments and Partial Reputation

Agents rarely act in isolation -- their behavioral history, in particula...

Please sign up or login with your details

Forgot password? Click here to reset