Zeroth-Order Algorithms for Nonconvex Minimax Problems with Improved Complexities

01/22/2020
by   Zhongruo Wang, et al.
0

In this paper, we study zeroth-order algorithms for minimax optimization problems that are nonconvex in one variable and strongly-concave in the other variable. Such minimax optimization problems have attracted significant attention lately due to their applications in modern machine learning tasks. We first design and analyze the Zeroth-Order Gradient Descent Ascent (ZO-GDA) algorithm, and provide improved results compared to existing works, in terms of oracle complexity. Next, we propose the Zeroth-Order Gradient Descent Multi-Step Ascent (ZO-GDMSA) algorithm that significantly improves the oracle complexity of ZO-GDA. We also provide stochastic version of ZO-GDA and ZO-GDMSA to handle stochastic nonconvex minimax problems, and provide oracle complexity results.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/24/2022

Zeroth-Order Alternating Gradient Descent Ascent Algorithms for a Class of Nonconvex-Nonconcave Minimax Problems

In this paper, we consider a class of nonconvex-nonconcave minimax probl...
research
03/09/2022

Federated Minimax Optimization: Improved Convergence Analyses and Algorithms

In this paper, we consider nonconvex minimax optimization, which is gain...
research
02/13/2022

Minimax in Geodesic Metric Spaces: Sion's Theorem and Algorithms

Determining whether saddle points exist or are approximable for nonconve...
research
09/22/2022

Nonsmooth Composite Nonconvex-Concave Minimax Optimization

Nonconvex-concave minimax optimization has received intense interest in ...
research
06/22/2022

Projection-free Constrained Stochastic Nonconvex Optimization with State-dependent Markov Data

We study a projection-free conditional gradient-type algorithm for const...
research
02/09/2021

Proximal Gradient Descent-Ascent: Variable Convergence under KŁ Geometry

The gradient descent-ascent (GDA) algorithm has been widely applied to s...
research
06/27/2020

Hybrid Variance-Reduced SGD Algorithms For Nonconvex-Concave Minimax Problems

We develop a novel variance-reduced algorithm to solve a stochastic nonc...

Please sign up or login with your details

Forgot password? Click here to reset