Nonconvex-Nonconcave Min-Max Optimization with a Small Maximization Domain

10/08/2021
by   Dmitrii M. Ostrovskii, et al.
0

We study the problem of finding approximate first-order stationary points in optimization problems of the form min_x ∈ Xmax_y ∈ Y f(x,y), where the sets X,Y are convex and Y is compact. The objective function f is smooth, but assumed neither convex in x nor concave in y. Our approach relies upon replacing the function f(x,·) with its kth order Taylor approximation (in y) and finding a near-stationary point in the resulting surrogate problem. To guarantee its success, we establish the following result: let the Euclidean diameter of Y be small in terms of the target accuracy ε, namely O(ε^2/k+1) for k ∈ℕ and O(ε) for k = 0, with the constant factors controlled by certain regularity parameters of f; then any ε-stationary point in the surrogate problem remains O(ε)-stationary for the initial problem. Moreover, we show that these upper bounds are nearly optimal: the aforementioned reduction provably fails when the diameter of Y is larger. For 0 ≤ k ≤ 2 the surrogate function can be efficiently maximized in y; our general approximation result then leads to efficient algorithms for finding a near-stationary point in nonconvex-nonconcave min-max problems, for which we also provide convergence guarantees.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/04/2018

Non-Convex Min-Max Optimization: Provable Algorithms and Applications in Machine Learning

Min-max saddle-point problems have broad applications in many tasks in m...
research
04/18/2021

Complexity Lower Bounds for Nonconvex-Strongly-Concave Min-Max Optimization

We provide a first-order oracle complexity lower bound for finding stati...
research
05/25/2022

Complexity-Optimal and Curvature-Free First-Order Methods for Finding Stationary Points of Composite Optimization Problems

This paper develops and analyzes an accelerated proximal descent method ...
research
09/21/2020

The Complexity of Constrained Min-Max Optimization

Despite its important applications in Machine Learning, min-max optimiza...
research
10/31/2020

Efficient Methods for Structured Nonconvex-Nonconcave Min-Max Optimization

The use of min-max optimization in adversarial training of deep neural n...
research
02/10/2020

On Complexity of Finding Stationary Points of Nonsmooth Nonconvex Functions

We provide the first non-asymptotic analysis for finding stationary poin...
research
05/25/2021

Practical Schemes for Finding Near-Stationary Points of Convex Finite-Sums

The problem of finding near-stationary points in convex optimization has...

Please sign up or login with your details

Forgot password? Click here to reset