On Adapting Nesterov's Scheme to Accelerate Iterative Methods for Linear Problems

by   Tao Hong, et al.

Nesterov's well-known scheme for accelerating gradient descent in convex optimization problems is adapted to accelerating stationary iterative solvers for linear systems. Compared with classical Krylov subspace acceleration methods, the proposed scheme requires more iterations, but it is trivial to implement and retains essentially the same computational cost as the unaccelerated method. An explicit formula for a fixed optimal parameter is derived in the case where the stationary iteration matrix has only real eigenvalues, based only on the smallest and largest eigenvalues. The fixed parameter, and corresponding convergence factor, are shown to maintain their optimality when the iteration matrix also has complex eigenvalues that are contained within an explicitly defined disk in the complex plane. A comparison to Chebyshev acceleration based on the same information of the smallest and largest real eigenvalues (dubbed Restricted Information Chebyshev acceleration) demonstrates that Nesterov's scheme is more robust in the sense that it remains optimal over a larger domain when the iteration matrix does have some complex eigenvalues. Numerical tests validate the efficiency of the proposed scheme. This work generalizes and extends the results of [1, Lemmas 3.1 and 3.2 and Theorem 3.3].



There are no comments yet.


page 6

page 7


A Robust Iterative Scheme for Symmetric Indefinite Systems

We propose a two-level nested preconditioned iterative scheme for solvin...

Accelerated Additive Schwarz Methods for Convex Optimization with Adaptive Restart

Based on an observation that additive Schwarz methods for general convex...

Complex Eigenvalues for Binary Subdivision Schemes

Convergence properties of binary stationary subdivision schemes for curv...

Anderson Acceleration for Geometry Optimization and Physics Simulation

Many computer graphics problems require computing geometric shapes subje...

Accelerating Smooth Games by Manipulating Spectral Shapes

We use matrix iteration theory to characterize acceleration in smooth ga...

Computable structural formulas for the distribution of the β-Jacobi edge eigenvalues

The Jacobi ensemble is one of the classical ensembles of random matrix t...

Iterations for the Unitary Sign Decomposition and the Unitary Eigendecomposition

We construct fast, structure-preserving iterations for computing the sig...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.