SPIRAL: A Superlinearly Convergent Incremental Proximal Algorithm for Nonconvex Finite Sum Minimization

07/17/2022
by   Pourya Behmandpoor, et al.
0

We introduce SPIRAL, a SuPerlinearly convergent Incremental pRoximal ALgorithm, for solving nonconvex regularized finite sum problems under a relative smoothness assumption. In the spirit of SVRG and SARAH, each iteration of SPIRAL consists of an inner and an outer loop. It combines incremental and full (proximal) gradient updates with a linesearch. It is shown that when using quasi-Newton directions, superlinear convergence is attained under mild assumptions at the limit points. More importantly, thanks to said linesearch, global convergence is ensured while it is shown that unit stepsize will be eventually always accepted. Simulation results on different convex, nonconvex, and non-Lipschitz differentiable problems show that our algorithm as well as its adaptive variant are competitive to the state of the art.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/25/2021

A proximal-proximal majorization-minimization algorithm for nonconvex tuning-free robust regression problems

In this paper, we introduce a proximal-proximal majorization-minimizatio...
research
01/06/2023

A Levenberg-Marquardt Method for Nonsmooth Regularized Least Squares

We develop a Levenberg-Marquardt method for minimizing the sum of a smoo...
research
12/31/2015

Linear Convergence of Proximal Gradient Algorithm with Extrapolation for a Class of Nonconvex Nonsmooth Minimization Problems

In this paper, we study the proximal gradient algorithm with extrapolati...
research
02/15/2019

ProxSARAH: An Efficient Algorithmic Framework for Stochastic Composite Nonconvex Optimization

In this paper, we propose a new stochastic algorithmic framework to solv...
research
05/26/2023

Sharpened Lazy Incremental Quasi-Newton Method

We consider the finite sum minimization of n strongly convex and smooth ...
research
09/01/2011

Nonconvex proximal splitting: batch and incremental algorithms

Within the unmanageably large class of nonconvex optimization, we consid...
research
12/04/2018

A probabilistic incremental proximal gradient method

In this paper, we propose a probabilistic optimization method, named pro...

Please sign up or login with your details

Forgot password? Click here to reset