DeepAI
Log In Sign Up

A globally convergent fast iterative shrinkage-thresholding algorithm with a new momentum factor for single and multi-objective convex optimization

05/11/2022
by   Hiroki Tanabe, et al.
0

Convex-composite optimization, which minimizes an objective function represented by the sum of a differentiable function and a convex one, is widely used in machine learning and signal/image processing. Fast Iterative Shrinkage Thresholding Algorithm (FISTA) is a typical method for solving this problem and has a global convergence rate of O(1 / k^2). Recently, this has been extended to multi-objective optimization, together with the proof of the O(1 / k^2) global convergence rate. However, its momentum factor is classical, and the convergence of its iterates has not been proven. In this work, introducing some additional hyperparameters (a, b), we propose another accelerated proximal gradient method with a general momentum factor, which is new even for the single-objective cases. We show that our proposed method also has a global convergence rate of O(1/k^2) for any (a,b), and further that the generated sequence of iterates converges to a weak Pareto solution when a is positive, an essential property for the finite-time manifold identification. Moreover, we report numerical results with various (a,b), showing that some of these choices give better results than the classical momentum factors.

READ FULL TEXT

page 20

page 21

12/11/2021

Convergence Rate Analysis of Accelerated Forward-Backward Algorithm with Generalized Nesterov Momentum Scheme

Nesterov's accelerated forward-backward algorithm (AFBA) is an efficient...
08/07/2021

Variable metric extrapolation proximal iterative hard thresholding method for ℓ_0 minimization problem

In this paper, we consider the ℓ_0 minimization problem whose objective ...
10/14/2021

Additive Schwarz Methods for Convex Optimization with Backtracking

This paper presents a novel backtracking strategy for additive Schwarz m...
12/13/2022

Linear Convergence of ISTA and FISTA

In this paper, we revisit the class of iterative shrinkage-thresholding ...
11/09/2022

Extragradient with Positive Momentum is Optimal for Games with Cross-Shaped Jacobian Spectrum

The extragradient method has recently gained increasing attention, due t...
11/03/2022

Proximal Subgradient Norm Minimization of ISTA and FISTA

For first-order smooth optimization, the research on the acceleration ph...
12/26/2017

IHT dies hard: Provable accelerated Iterative Hard Thresholding

We study --both in theory and practice-- the use of momentum motions in ...

Code Repositories

zfista

A globally convergent fast iterative shrinkage-thresholding algorithm with a new momentum factor for single and multi-objective convex optimization


view repo