The Complexity of Nonconvex-Strongly-Concave Minimax Optimization

03/29/2021
by   Siqi Zhang, et al.
0

This paper studies the complexity for finding approximate stationary points of nonconvex-strongly-concave (NC-SC) smooth minimax problems, in both general and averaged smooth finite-sum settings. We establish nontrivial lower complexity bounds of Ω(√(κ)Δ Lϵ^-2) and Ω(n+√(nκ)Δ Lϵ^-2) for the two settings, respectively, where κ is the condition number, L is the smoothness constant, and Δ is the initial gap. Our result reveals substantial gaps between these limits and best-known upper bounds in the literature. To close these gaps, we introduce a generic acceleration scheme that deploys existing gradient-based methods to solve a sequence of crafted strongly-convex-strongly-concave subproblems. In the general setting, the complexity of our proposed algorithm nearly matches the lower bound; in particular, it removes an additional poly-logarithmic dependence on accuracy present in previous works. In the averaged smooth finite-sum setting, our proposed algorithm improves over previous algorithms by providing a nearly-tight dependence on the condition number.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/02/2019

Efficient Algorithms for Smooth Minimax Optimization

This paper studies first order methods for solving smooth minimax optimi...
research
06/11/2020

Improved Algorithms for Convex-Concave Minimax Optimization

This paper studies minimax optimization problems min_x max_y f(x,y), whe...
research
02/05/2020

Near-Optimal Algorithms for Minimax Optimization

This paper resolves a longstanding open question pertaining to the desig...
research
05/28/2022

Uniform Convergence and Generalization for Nonconvex Stochastic Minimax Problems

This paper studies the uniform convergence and generalization bounds for...
research
06/18/2020

Stochastic Variance Reduction via Accelerated Dual Averaging for Finite-Sum Optimization

In this paper, we introduce a simplified and unified method for finite-s...
research
04/05/2023

Decentralized gradient descent maximization method for composite nonconvex strongly-concave minimax problems

Minimax problems have recently attracted a lot of research interests. A ...
research
03/16/2023

Limit setting using spacings in the presence of unknown backgrounds

Finding upper limits on the rate of events from a proposed process in th...

Please sign up or login with your details

Forgot password? Click here to reset