AI Chat AI Image Generator AI Video Text to Speech

A Unified Analysis of Extra-gradient and Optimistic Gradient Methods for Saddle Point Problems: Proximal Point Approach

01/24/2019

∙

by Aryan Mokhtari, et al.

∙

∙

We consider solving convex-concave saddle point problems. We focus on two variants of gradient decent-ascent algorithms, Extra-gradient (EG) and Optimistic Gradient (OGDA) methods, and show that they admit a unified analysis as approximations of the classical proximal point method for solving saddle-point problems. This viewpoint enables us to generalize EG (in terms of extrapolation steps) and OGDA (in terms of parameters) and obtain new convergence rate results for these algorithms for the bilinear case as well as the strongly convex-concave case.

Aryan Mokhtari
41 publications
Asuman Ozdaglar
33 publications
Sarath Pattathil
12 publications

research

∙ 06/03/2019

Proximal Point Approximations Achieving a Convergence Rate of O(1/k) for Smooth Convex-Concave Saddle Point Problems: Optimistic Gradient and Extra-gradient Methods

In this paper we analyze the iteration complexity of the optimistic grad...

0 Aryan Mokhtari, et al. ∙

research

∙ 01/25/2021

Extragradient and Extrapolation Methods with Generalized Bregman Distances for Saddle Point Problems

In this work, we introduce two algorithmic frameworks, named Bregman ext...

0 Hui Zhang, et al. ∙

research

∙ 01/23/2020

An O(s^r)-Resolution ODE Framework for Discrete-Time Optimization Algorithms and Applications to Convex-Concave Saddle-Point Problems

There has been a long history of using Ordinary Differential Equations (...

0 Haihao Lu, et al. ∙

research

∙ 10/17/2022

Tight Analysis of Extra-gradient and Optimistic Gradient Methods For Nonconvex Minimax Problems

Despite the established convergence theory of Optimistic Gradient Descen...

0 Pouria Mahdavinia, et al. ∙

research

∙ 09/13/2019

A Stochastic Proximal Point Algorithm for Saddle-Point Problems

We consider saddle point problems which objective functions are the aver...

0 Luo Luo, et al. ∙

research

∙ 05/17/2020

From Proximal Point Method to Nesterov's Acceleration

The proximal point method (PPM) is a fundamental method in optimization ...

0 Kwangjun Ahn, et al. ∙

research

∙ 07/07/2018

Mirror descent in saddle-point problems: Going the extra (gradient) mile

Owing to their connection with generative adversarial networks (GANs), s...

0 Panayotis Mertikopoulos, et al. ∙