A globally convergent fast iterative shrinkage-thresholding algorithm with a new momentum factor for single and multi-objective convex optimization
share
Convex-composite optimization, which minimizes an objective function represented by the sum of a differentiable function and a convex one, is widely used in machine learning and signal/image processing. Fast Iterative Shrinkage Thresholding Algorithm (FISTA) is a typical method for solving this problem and has a global convergence rate of O(1 / k^2). Recently, this has been extended to multi-objective optimization, together with the proof of the O(1 / k^2) global convergence rate. However, its momentum factor is classical, and the convergence of its iterates has not been proven. In this work, introducing some additional hyperparameters (a, b), we propose another accelerated proximal gradient method with a general momentum factor, which is new even for the single-objective cases. We show that our proposed method also has a global convergence rate of O(1/k^2) for any (a,b), and further that the generated sequence of iterates converges to a weak Pareto solution when a is positive, an essential property for the finite-time manifold identification. Moreover, we report numerical results with various (a,b), showing that some of these choices give better results than the classical momentum factors.
READ FULL TEXT