Discretization Drift in Two-Player Games

by   Mihaela Rosca, et al.

Gradient-based methods for two-player games produce rich dynamics that can solve challenging problems, yet can be difficult to stabilize and understand. Part of this complexity originates from the discrete update steps given by simultaneous or alternating gradient descent, which causes each player to drift away from the continuous gradient flow – a phenomenon we call discretization drift. Using backward error analysis, we derive modified continuous dynamical systems that closely follow the discrete dynamics. These modified dynamics provide an insight into the notorious challenges associated with zero-sum games, including Generative Adversarial Networks. In particular, we identify distinct components of the discretization drift that can alter performance and in some cases destabilize the game. Finally, quantifying discretization drift allows us to identify regularizers that explicitly cancel harmful forms of drift or strengthen beneficial forms of drift, and thus improve performance of GAN training.


page 32

page 36

page 37


Implicit regularisation in stochastic gradient descent: from single-objective to two-player games

Recent years have seen many insights on deep learning optimisation being...

Alternating Mirror Descent for Constrained Min-Max Games

In this paper we study two-player bilinear zero-sum games with constrain...

Game Theoretic Optimization via Gradient-based Nikaido-Isoda Function

Computing Nash equilibrium (NE) of multi-player games has witnessed rene...

A Finite-Sample Analysis of Payoff-Based Independent Learning in Zero-Sum Stochastic Games

We study two-player zero-sum stochastic games, and propose a form of ind...

Interaction Matters: A Note on Non-asymptotic Local Convergence of Generative Adversarial Networks

Motivated by the pursuit of a systematic computational and algorithmic u...

Constants of Motion: The Antidote to Chaos in Optimization and Game Dynamics

Several recent works in online optimization and game dynamics have estab...

Supervised Seeded Iterated Learning for Interactive Language Learning

Language drift has been one of the major obstacles to train language mod...

Please sign up or login with your details

Forgot password? Click here to reset