Efficient Methods for Structured Nonconvex-Nonconcave Min-Max Optimization

10/31/2020
by   Jelena Diakonikolas, et al.
20

The use of min-max optimization in adversarial training of deep neural network classifiers and training of generative adversarial networks has motivated the study of nonconvex-nonconcave optimization objectives, which frequently arise in these applications. Unfortunately, recent results have established that even approximate first-order stationary points of such objectives are intractable, even under smoothness conditions, motivating the study of min-max objectives with additional structure. We introduce a new class of structured nonconvex-nonconcave min-max optimization problems, proposing a generalization of the extragradient algorithm which provably converges to a stationary point. The algorithm applies not only to Euclidean spaces, but also to general ℓ_p-normed finite-dimensional real vector spaces. We also discuss its stability under stochastic oracles and provide bounds on its sample complexity. Our iteration complexity and sample complexity bounds either match or improve the best known bounds for the same or less general nonconvex-nonconcave settings, such as those that satisfy variational coherence or in which a weak solution to the associated variational inequality problem is assumed to exist.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/21/2020

The Complexity of Constrained Min-Max Optimization

Despite its important applications in Machine Learning, min-max optimiza...
research
10/29/2020

A Single-Loop Smoothed Gradient Descent-Ascent Algorithm for Nonconvex-Concave Min-Max Problems

Nonconvex-concave min-max problem arises in many machine learning applic...
research
10/18/2022

STay-ON-the-Ridge: Guaranteed Convergence to Local Minimax Equilibrium in Nonconvex-Nonconcave Games

Min-max optimization problems involving nonconvex-nonconcave objectives ...
research
10/08/2021

Nonconvex-Nonconcave Min-Max Optimization with a Small Maximization Domain

We study the problem of finding approximate first-order stationary point...
research
02/10/2020

On Complexity of Finding Stationary Points of Nonsmooth Nonconvex Functions

We provide the first non-asymptotic analysis for finding stationary poin...
research
06/16/2020

Enhanced First and Zeroth Order Variance Reduced Algorithms for Min-Max Optimization

Min-max optimization captures many important machine learning problems s...
research
10/26/2020

LEAD: Least-Action Dynamics for Min-Max Optimization

Adversarial formulations such as generative adversarial networks (GANs) ...

Please sign up or login with your details

Forgot password? Click here to reset