Gradient-Free Methods for Saddle-Point Problem

05/12/2020
by   Aleksandr Beznosikov, et al.
0

In the paper, we generalize the approach Gasnikov et. al, 2017, which allows to solve (stochastic) convex optimization problems with an inexact gradient-free oracle, to the convex-concave saddle-point problem. The proposed approach works, at least, like the best existing approaches. But for a special set-up (simplex type constraints and closeness of Lipschitz constants in 1 and 2 norms) our approach reduces n/log n times the required number of oracle calls (function calculations). Our method uses a stochastic approximation of the gradient via finite differences. In this case, the function must be specified not only on the optimization set itself, but in a certain neighbourhood of it. In the second part of the paper, we analyze the case when such an assumption cannot be made, we propose a general approach on how to modernize the method to solve this problem, and also we apply this approach to particular cases of some classical sets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/15/2023

Projection-Free Methods for Stochastic Simple Bilevel Optimization with Convex Lower-level Problem

In this paper, we study a class of stochastic bilevel optimization probl...
research
08/07/2022

Decomposable Non-Smooth Convex Optimization with Nearly-Linear Gradient Oracle Complexity

Many fundamental problems in machine learning can be formulated by the c...
research
09/21/2020

Zeroth-Order Algorithms for Smooth Saddle-Point Problems

In recent years, the importance of saddle-point problems in machine lear...
research
01/21/2020

SA vs SAA for population Wasserstein barycenter calculation

In Machine Learning and Optimization community there are two main approa...
research
02/01/2022

Decentralized Stochastic Variance Reduced Extragradient Method

This paper studies decentralized convex-concave minimax optimization pro...
research
06/20/2023

Globally optimal solutions to a class of fractional optimization problems based on proximity gradient algorithm

We establish globally optimal solutions to a class of fractional optimiz...
research
10/05/2020

Projection Efficient Subgradient Method and Optimal Nonsmooth Frank-Wolfe Method

We consider the classical setting of optimizing a nonsmooth Lipschitz co...

Please sign up or login with your details

Forgot password? Click here to reset