A J-Symmetric Quasi-Newton Method for Minimax Problems

02/04/2022
by   Azam Asl, et al.
0

Minimax problems have gained tremendous attentions across the optimization and machine learning community recently. In this paper, we introduce a new quasi-Newton method for minimax problems, which we call J-symmetric quasi-Newton method. The method is obtained by exploiting the J-symmetric structure of the second-order derivative of the objective function in minimax problem. We show that the Hessian estimation (as well as its inverse) can be updated by a rank-2 operation, and it turns out that the update rule is a natural generalization of the classic Powell symmetric Broyden (PSB) method from minimization problems to minimax problems. In theory, we show that our proposed quasi-Newton algorithm enjoys local Q-superlinear convergence to a desirable solution under standard regularity conditions. Furthermore, we introduce a trust-region variant of the algorithm that enjoys global R-superlinear convergence. Finally, we present numerical experiments that verify our theory and show the effectiveness of our proposed algorithms compared to Broyden's method and the extragradient method on three classes of minimax problems.

READ FULL TEXT

page 1

page 2

page 3

page 4

11/06/2020

Generalization of Quasi-Newton Methods: Application to Robust Symmetric Multisecant Updates

Quasi-Newton techniques approximate the Newton step by estimating the He...
12/26/2019

Quasi-Newton Trust Region Policy Optimization

We propose a trust region method for policy optimization that employs Qu...
06/25/2020

Newton-type Methods for Minimax Optimization

Differential games, in particular two-player sequential games (a.k.a. mi...
10/01/2020

Single-stage gradient-based stellarator coil design: Optimization for near-axis quasi-symmetry

We present a new coil design paradigm for magnetic confinement in stella...
02/01/2016

A Quasi-Bayesian Perspective to Online Clustering

When faced with high frequency streams of data, clustering raises theore...
07/13/2021

A New Multipoint Symmetric Secant Method with a Dense Initial Matrix

In large-scale optimization, when either forming or storing Hessian matr...
07/29/2022

Compact representations of structured BFGS matrices

For general large-scale optimization problems compact representations ex...