An optimal scheduled learning rate for a randomized Kaczmarz algorithm

02/24/2022

∙

by   Nicholas F. Marshall, et al.

∙

0

∙

share

We study how the learning rate affects the performance of a relaxed randomized Kaczmarz algorithm for solving A x ≈ b + ε, where A x =b is a consistent linear system and ε has independent mean zero random entries. We derive a scheduled learning rate which optimizes a bound on the expected error that is sharp in certain cases; in contrast to the exponential convergence of the standard randomized Kaczmarz algorithm, our optimized bound involves the reciprocal of the Lambert-W function of an exponential.