The linear convergence of the greedy randomized Kaczmarz method is deterministic

by   Yansheng Su, et al.

To improve the convergence property of the randomized Kaczmarz (RK) method for solving linear systems, Bai and Wu (SIAM J. Sci. Comput., 40(1):A592–A606, 2018) originally introduced a greedy probability criterion for effectively selecting the working row from the coefficient matrix and constructed the greedy randomized Kaczmarz (GRK) method. Due to its simplicity and efficiency, this approach has inspired numerous subsequent works in recent years, such as the capped adaptive sampling rule, the greedy augmented randomized Kaczmarz method, and the greedy randomized coordinate descent method. Since the iterates of the GRK method are actually random variables, existing convergence analyses are all related to the expectation of the error. In this note, we prove that the linear convergence rate of the GRK method is deterministic, i.e. not in the sense of expectation. Moreover, the Polyak's heavy ball momentum technique is incorporated to improve the performance of the GRK method. We propose a refined convergence analysis, compared with the technique used in Loizou and Richtárik (Comput. Optim. Appl., 77(3):653–710, 2020), of momentum variants of randomized iterative methods, which shows that the proposed GRK method with momentum (mGRK) also enjoys a deterministic linear convergence. Numerical experiments show that the mGRK method is more efficient than the GRK method.


page 1

page 2

page 3

page 4


Sketch Project Methods for Linear Feasibility Problems: Greedy Sampling Momentum

We develop two greedy sampling rules for the Sketch Project method f...

Stochastic Steepest Descent Methods for Linear Systems: Greedy Sampling Momentum

Recently proposed adaptive Sketch Project (SP) methods connect sever...

A Kaczmarz Method with Simple Random Sampling for Solving Large Linear Systems

The Kaczmarz method is a popular iterative scheme for solving large-scal...

Restarted randomized surrounding methods for solving large linear equations

A class of restarted randomized surrounding methods are presented to acc...

On the Polyak momentum variants of the greedy deterministic single and multiple row-action methods

For solving a consistent system of linear equations, the classical row-a...

On the relaxed greedy randomized Kaczmarz methods with momentum acceleration for solving matrix equation AXB=C

With the growth of data, it is more important than ever to develop an ef...

Greed Works: An Improved Analysis of Sampling Kaczmarz-Motkzin

Stochastic iterative algorithms have gained recent interest in machine l...

Please sign up or login with your details

Forgot password? Click here to reset