A Momentum Accelerated Adaptive Cubic Regularization Method for Nonconvex Optimization

10/12/2022
by   Yihang Gao, et al.
0

The cubic regularization method (CR) and its adaptive version (ARC) are popular Newton-type methods in solving unconstrained non-convex optimization problems, due to its global convergence to local minima under mild conditions. The main aim of this paper is to develop a momentum-accelerated adaptive cubic regularization method (ARCm) to improve the convergent performance. With the proper choice of momentum step size, we show the global convergence of ARCm and the local convergence can also be guaranteed under the property. Such global and local convergence can also be established when inexact solvers with low computational costs are employed in the iteration procedure. Numerical results for non-convex logistic regression and robust linear regression models are reported to demonstrate that the proposed ARCm significantly outperforms state-of-the-art cubic regularization methods (e.g., CR, momentum-based CR, ARC) and the trust region method. In particular, the number of iterations required by ARCm is less than 10% to 50% required by the most competitive method (ARC) in the experiments.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/13/2018

Stochastic Variance-Reduced Cubic Regularized Newton Method

We propose a stochastic variance-reduced cubic regularized Newton method...
research
10/09/2018

Cubic Regularization with Momentum for Nonconvex Optimization

Momentum is a popular technique to accelerate the convergence in practic...
research
10/14/2021

Additive Schwarz Methods for Convex Optimization with Backtracking

This paper presents a novel backtracking strategy for additive Schwarz m...
research
05/16/2017

Sub-sampled Cubic Regularization for Non-convex Optimization

We consider the minimization of non-convex functions that typically aris...
research
10/04/2020

Quickly Finding a Benign Region via Heavy Ball Momentum in Non-Convex Optimization

The Heavy Ball Method, proposed by Polyak over five decades ago, is a fi...
research
10/15/2019

Adaptive Step Sizes in Variance Reduction via Regularization

The main goal of this work is equipping convex and nonconvex problems wi...
research
05/30/2023

Adaptive Quasi-Newton and Anderson Acceleration Framework with Explicit Global (Accelerated) Convergence Rates

Despite the impressive numerical performance of quasi-Newton and Anderso...

Please sign up or login with your details

Forgot password? Click here to reset