Online Optimization in Games via Control Theory: Connecting Regret, Passivity and Poincaré Recurrence

06/09/2021
by   Yun Kuen Cheung, et al.
0

We present a novel control-theoretic understanding of online optimization and learning in games, via the notion of passivity. Passivity is a fundamental concept in control theory, which abstracts energy conservation and dissipation in physical systems. It has become a standard tool in analysis of general feedback systems, to which game dynamics belong. Our starting point is to show that all continuous-time Follow-the-Regularized-Leader (FTRL) dynamics, which include the well-known Replicator Dynamic, are lossless, i.e. it is passive with no energy dissipation. Interestingly, we prove that passivity implies bounded regret, connecting two fundamental primitives of control theory and online optimization. The observation of energy conservation in FTRL inspires us to present a family of lossless learning dynamics, each of which has an underlying energy function with a simple gradient structure. This family is closed under convex combination; as an immediate corollary, any convex combination of FTRL dynamics is lossless and thus has bounded regret. This allows us to extend the framework of Fox and Shamma [Games, 2013] to prove not just global asymptotic stability results for game dynamics, but Poincaré recurrence results as well. Intuitively, when a lossless game (e.g. graphical constant-sum game) is coupled with lossless learning dynamics, their feedback interconnection is also lossless, which results in a pendulum-like energy-preserving recurrent behavior, generalizing the results of Piliouras and Shamma [SODA, 2014] and Mertikopoulos, Papadimitriou and Piliouras [SODA, 2018].

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

03/05/2019

Multi-Agent Learning in Network Zero-Sum Games is a Hamiltonian System

Zero-sum games are natural, if informal, analogues of closed physical sy...
04/25/2022

Uncoupled Learning Dynamics with O(log T) Swap Regret in Multiplayer Games

In this paper we establish efficient and uncoupled learning dynamics so ...
07/28/2020

Faster Game Solving via Predictive Blackwell Approachability: Connecting Regret Matching and Mirror Descent

Blackwell approachability is a framework for reasoning about repeated ga...
06/28/2021

Evolutionary Dynamics and Φ-Regret Minimization in Games

Regret has been established as a foundational concept in online learning...
10/19/2017

Online Monotone Games

Algorithmic game theory (AGT) focuses on the design and analysis of algo...
07/04/2021

Learning in nonatomic games, Part I: Finite action spaces and population games

We examine the long-run behavior of a wide range of dynamics for learnin...
09/08/2021

Constants of Motion: The Antidote to Chaos in Optimization and Game Dynamics

Several recent works in online optimization and game dynamics have estab...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.