A J-Symmetric Quasi-Newton Method for Minimax Problems

by   Azam Asl, et al.

Minimax problems have gained tremendous attentions across the optimization and machine learning community recently. In this paper, we introduce a new quasi-Newton method for minimax problems, which we call J-symmetric quasi-Newton method. The method is obtained by exploiting the J-symmetric structure of the second-order derivative of the objective function in minimax problem. We show that the Hessian estimation (as well as its inverse) can be updated by a rank-2 operation, and it turns out that the update rule is a natural generalization of the classic Powell symmetric Broyden (PSB) method from minimization problems to minimax problems. In theory, we show that our proposed quasi-Newton algorithm enjoys local Q-superlinear convergence to a desirable solution under standard regularity conditions. Furthermore, we introduce a trust-region variant of the algorithm that enjoys global R-superlinear convergence. Finally, we present numerical experiments that verify our theory and show the effectiveness of our proposed algorithms compared to Broyden's method and the extragradient method on three classes of minimax problems.


page 1

page 2

page 3

page 4


Generalization of Quasi-Newton Methods: Application to Robust Symmetric Multisecant Updates

Quasi-Newton techniques approximate the Newton step by estimating the He...

Quasi-Newton Trust Region Policy Optimization

We propose a trust region method for policy optimization that employs Qu...

Newton-type Methods for Minimax Optimization

Differential games, in particular two-player sequential games (a.k.a. mi...

Single-stage gradient-based stellarator coil design: Optimization for near-axis quasi-symmetry

We present a new coil design paradigm for magnetic confinement in stella...

A Quasi-Bayesian Perspective to Online Clustering

When faced with high frequency streams of data, clustering raises theore...

A New Multipoint Symmetric Secant Method with a Dense Initial Matrix

In large-scale optimization, when either forming or storing Hessian matr...

Compact representations of structured BFGS matrices

For general large-scale optimization problems compact representations ex...