DeepAI

# A Newton-CG based augmented Lagrangian method for finding a second-order stationary point of nonconvex equality constrained optimization with complexity guarantees

In this paper we consider finding a second-order stationary point (SOSP) of nonconvex equality constrained optimization when a nearly feasible point is known. In particular, we first propose a new Newton-CG method for finding an approximate SOSP of unconstrained optimization and show that it enjoys a substantially better complexity than the Newton-CG method [56]. We then propose a Newton-CG based augmented Lagrangian (AL) method for finding an approximate SOSP of nonconvex equality constrained optimization, in which the proposed Newton-CG method is used as a subproblem solver. We show that under a generalized linear independence constraint qualification (GLICQ), our AL method enjoys a total inner iteration complexity of O(ϵ^-7/2) and an operation complexity of O(ϵ^-7/2min{n,ϵ^-3/4}) for finding an (ϵ,√(ϵ))-SOSP of nonconvex equality constrained optimization with high probability, which are significantly better than the ones achieved by the proximal AL method [60]. Besides, we show that it has a total inner iteration complexity of O(ϵ^-11/2) and an operation complexity of O(ϵ^-11/2min{n,ϵ^-5/4}) when the GLICQ does not hold. To the best of our knowledge, all the complexity results obtained in this paper are new for finding an approximate SOSP of nonconvex equality constrained optimization with high probability. Preliminary numerical results also demonstrate the superiority of our proposed methods over the ones in [56,60].

• 5 publications
• 25 publications
• 12 publications
01/10/2023

### A Newton-CG based barrier-augmented Lagrangian method for general nonconvex conic optimization

In this paper we consider finding an approximate second-order stationary...
07/12/2022

### A Newton-CG based barrier method for finding a second-order stationary point of nonconvex conic optimization with complexity guarantees

In this paper we consider finding an approximate second-order stationary...
02/29/2020

### OpEn: Code Generation for Embedded Nonconvex Optimization

We present Optimization Engine (OpEn): an open-source code generation to...
03/19/2020

### Augmented Lagrangian based first-order methods for convex and nonconvex programs: nonergodic convergence and iteration complexity

First-order methods (FOMs) have been widely used for large-scale problem...
05/23/2022

### Augmented Newton Method for Optimization: Global Linear Rate and Momentum Interpretation

We propose two variants of Newton method for solving unconstrained minim...
05/20/2017

### Optimality of orders one to three and beyond: characterization and evaluation complexity in constrained nonconvex optimization

Necessary conditions for high-order optimality in smooth nonlinear const...
02/17/2020

### Stochastic Gauss-Newton Algorithms for Nonconvex Compositional Optimization

We develop two new stochastic Gauss-Newton algorithms for solving a clas...