On the Suboptimality of Negative Momentum for Minimax Optimization

08/17/2020
by   Guodong Zhang, et al.
0

Smooth game optimization has recently attracted great interest in machine learning as it generalizes the single-objective optimization paradigm. However, game dynamics is more complex due to the interaction between different players and is therefore fundamentally different from minimization, posing new challenges for algorithm design. Notably, it has been shown that negative momentum is preferred due to its ability of reducing oscillation in game dynamics. Nevertheless, existing analysis about negative momentum was restricted to simple bilinear games. In this paper, we extend the analysis of negative momentum to smooth and strongly-convex strongly-concave minimax games by taking the variational inequality formulation. By connecting momentum method with Chebyshev polynomials, we show that negative momentum accelerates convergence of game dynamics locally, though with a suboptimal rate. To the best of our knowledge, this is the first work that provides an explicit convergence rate for negative momentum in this setting.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/12/2018

Negative Momentum for Improved Game Dynamics

Games generalize the optimization paradigm by introducing different obje...
research
06/23/2021

Understanding Modern Techniques in Optimization: Frank-Wolfe, Nesterov's Momentum, and Polyak's Momentum

In the first part of this dissertation research, we develop a modular fr...
research
11/09/2022

Extragradient with Positive Momentum is Optimal for Games with Cross-Shaped Jacobian Spectrum

The extragradient method has recently gained increasing attention, due t...
research
03/20/2019

Is Basketball a Game of Runs?

Basketball is often referred to as "a game of runs." We investigate the ...
research
05/27/2019

Revisiting Stochastic Extragradient

We consider a new extension of the extragradient method that is motivate...
research
01/02/2020

Accelerating Smooth Games by Manipulating Spectral Shapes

We use matrix iteration theory to characterize acceleration in smooth ga...
research
09/12/2020

Hot Racquet or Not? An Exploration of Momentum in Grand Slam Tennis Matches

The presence of momentum in sports, where the outcome of a previous even...

Please sign up or login with your details

Forgot password? Click here to reset