The Mechanics of n-Player Differentiable Games

02/15/2018
by   David Balduzzi, et al.
0

The cornerstone underpinning deep learning is the guarantee that gradient descent on an objective converges to local minima. Unfortunately, this guarantee fails in settings, such as generative adversarial nets, where there are multiple interacting losses. The behavior of gradient-based methods in games is not well understood -- and is becoming increasingly important as adversarial and multi-objective architectures proliferate. In this paper, we develop new techniques to understand and control the dynamics in general games. The key result is to decompose the second-order dynamics into two components. The first is related to potential games, which reduce to gradient descent on an implicit function; the second relates to Hamiltonian games, a new class of games that obey a conservation law, akin to conservation laws in classical mechanical systems. The decomposition motivates Symplectic Gradient Adjustment (SGA), a new algorithm for finding stable fixed points in general games. Basic experiments show SGA is competitive with recently proposed algorithms for finding local Nash equilibria in GANs -- whilst at the same time being applicable to -- and having guarantees in -- much more general games.

READ FULL TEXT

page 8

page 12

page 13

page 17

research
05/13/2019

Differentiable Game Mechanics

Deep learning is built on the foundational guarantee that gradient desce...
research
11/16/2021

Polymatrix Competitive Gradient Descent

Many economic games and machine learning approaches can be cast as compe...
research
05/15/2019

Game Theoretic Optimization via Gradient-based Nikaido-Isoda Function

Computing Nash equilibrium (NE) of multi-player games has witnessed rene...
research
04/16/2018

On the Convergence of Competitive, Multi-Agent Gradient-Based Learning

As learning algorithms are increasingly deployed in markets and other co...
research
01/14/2020

Smooth markets: A basic mechanism for organizing gradient-based learners

With the success of modern machine learning, it is becoming increasingly...
research
07/11/2023

Implicit regularisation in stochastic gradient descent: from single-objective to two-player games

Recent years have seen many insights on deep learning optimisation being...
research
11/10/2021

Training Generative Adversarial Networks with Adaptive Composite Gradient

The wide applications of Generative adversarial networks benefit from th...

Please sign up or login with your details

Forgot password? Click here to reset