On the Impossibility of Global Convergence in Multi-Loss Optimization

05/26/2020
by   Alistair Letcher, et al.
0

Under mild regularity conditions, gradient-based methods converge globally to a critical point in the single-loss setting. This is known to break down for vanilla gradient descent when moving to multi-loss optimization, but can we hope to build some algorithm with global guarantees? We negatively resolve this open problem by proving that any reasonable algorithm will exhibit limit cycles or diverge to infinite losses in some differentiable game, even in two-player games with zero-sum interactions. A reasonable algorithm is simply one which avoids strict maxima, an exceedingly weak assumption since converging to maxima would be the opposite of minimization. This impossibility theorem holds even if we impose existence of a strict minimum and no other critical points. The proof is constructive, enabling us to display explicit limit cycles for existing gradient-based methods. Nonetheless, it remains an open question whether cycles arise in high-dimensional games of interest to ML practitioners, such as GANs or multi-agent RL.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/16/2018

On the Convergence of Competitive, Multi-Agent Gradient-Based Learning

As learning algorithms are increasingly deployed in markets and other co...
research
05/27/2022

Competitive Gradient Optimization

We study the problem of convergence to a stationary point in zero-sum ga...
research
10/23/2022

Symmetric (Optimistic) Natural Policy Gradient for Multi-agent Learning with Parameter Convergence

Multi-agent interactions are increasingly important in the context of re...
research
01/29/2021

Poincaré-Bendixson Limit Sets in Multi-Agent Learning

A key challenge of evolutionary game theory and multi-agent learning is ...
research
05/13/2019

Differentiable Game Mechanics

Deep learning is built on the foundational guarantee that gradient desce...
research
09/30/2020

Gradient Descent-Ascent Provably Converges to Strict Local Minmax Equilibria with a Finite Timescale Separation

We study the role that a finite timescale separation parameter τ has on ...
research
02/20/2023

Escaping limit cycles: Global convergence for constrained nonconvex-nonconcave minimax problems

This paper introduces a new extragradient-type algorithm for a class of ...

Please sign up or login with your details

Forgot password? Click here to reset