Doubly Smoothed GDA: Global Convergent Algorithm for Constrained Nonconvex-Nonconcave Minimax Optimization

12/26/2022
by   Taoli Zheng, et al.
0

Nonconvex-nonconcave minimax optimization has been the focus of intense research over the last decade due to its broad applications in machine learning and operation research. Unfortunately, most existing algorithms cannot be guaranteed to converge and always suffer from limit cycles. Their global convergence relies on certain conditions that are difficult to check, including but not limited to the global Polyak-Łojasiewicz condition, the existence of a solution satisfying the weak Minty variational inequality and α-interaction dominant condition. In this paper, we develop the first provably convergent algorithm called doubly smoothed gradient descent ascent method, which gets rid of the limit cycle without requiring any additional conditions. We further show that the algorithm has an iteration complexity of 𝒪(ϵ^-4) for finding a game stationary point, which matches the best iteration complexity of single-loop algorithms under nonconcave-concave settings. The algorithm presented here opens up a new path for designing provable algorithms for nonconvex-nonconcave minimax optimization problems.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/02/2019

On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems

We consider nonconvex-concave minimax problems, _x_y∈Y f(x, y), where f ...
research
02/20/2023

Escaping limit cycles: Global convergence for constrained nonconvex-nonconcave minimax problems

This paper introduces a new extragradient-type algorithm for a class of ...
research
06/15/2020

The Landscape of Nonconvex-Nonconcave Minimax Optimization

Minimax optimization has become a central tool for modern machine learni...
research
03/07/2023

Enhanced Adaptive Gradient Algorithms for Nonconvex-PL Minimax Optimization

In the paper, we study a class of nonconvex nonconcave minimax optimizat...
research
09/22/2022

Nonsmooth Composite Nonconvex-Concave Minimax Optimization

Nonconvex-concave minimax optimization has received intense interest in ...
research
10/20/2020

Limiting Behaviors of Nonconvex-Nonconcave Minimax Optimization via Continuous-Time Systems

Unlike nonconvex optimization, where gradient descent is guaranteed to c...
research
01/30/2018

An Incremental Path-Following Splitting Method for Linearly Constrained Nonconvex Nonsmooth Programs

The linearly constrained nonconvex nonsmooth program has drawn much atte...

Please sign up or login with your details

Forgot password? Click here to reset