A globally convergent fast iterative shrinkage-thresholding algorithm with a new momentum factor for single and multi-objective convex optimization

05/11/2022
by   Hiroki Tanabe, et al.
0

Convex-composite optimization, which minimizes an objective function represented by the sum of a differentiable function and a convex one, is widely used in machine learning and signal/image processing. Fast Iterative Shrinkage Thresholding Algorithm (FISTA) is a typical method for solving this problem and has a global convergence rate of O(1 / k^2). Recently, this has been extended to multi-objective optimization, together with the proof of the O(1 / k^2) global convergence rate. However, its momentum factor is classical, and the convergence of its iterates has not been proven. In this work, introducing some additional hyperparameters (a, b), we propose another accelerated proximal gradient method with a general momentum factor, which is new even for the single-objective cases. We show that our proposed method also has a global convergence rate of O(1/k^2) for any (a,b), and further that the generated sequence of iterates converges to a weak Pareto solution when a is positive, an essential property for the finite-time manifold identification. Moreover, we report numerical results with various (a,b), showing that some of these choices give better results than the classical momentum factors.

READ FULL TEXT

page 20

page 21

research
12/11/2021

Convergence Rate Analysis of Accelerated Forward-Backward Algorithm with Generalized Nesterov Momentum Scheme

Nesterov's accelerated forward-backward algorithm (AFBA) is an efficient...
research
08/07/2021

Variable metric extrapolation proximal iterative hard thresholding method for ℓ_0 minimization problem

In this paper, we consider the ℓ_0 minimization problem whose objective ...
research
10/14/2021

Additive Schwarz Methods for Convex Optimization with Backtracking

This paper presents a novel backtracking strategy for additive Schwarz m...
research
12/13/2022

Linear Convergence of ISTA and FISTA

In this paper, we revisit the class of iterative shrinkage-thresholding ...
research
11/09/2022

Extragradient with Positive Momentum is Optimal for Games with Cross-Shaped Jacobian Spectrum

The extragradient method has recently gained increasing attention, due t...
research
12/26/2017

IHT dies hard: Provable accelerated Iterative Hard Thresholding

We study --both in theory and practice-- the use of momentum motions in ...
research
06/16/2023

Linear convergence of Nesterov-1983 with the strong convexity

For modern gradient-based optimization, a developmental landmark is Nest...

Please sign up or login with your details

Forgot password? Click here to reset