Solving A System Of Linear Equations By Randomized Orthogonal Projections

11/12/2021
by   Alireza Entezari, et al.
0

Randomization has shown catalyzing effects in linear algebra with promising perspectives for tackling computational challenges in large-scale problems. For solving a system of linear equations, we demonstrate the convergence of a broad class of algorithms that at each step pick a subset of n equations at random and update the iterate with the orthogonal projection to the subspace those equations represent. We identify, in this context, a specific degree-n polynomial that non-linearly transforms the singular values of the system towards equalization. This transformation to singular values and the corresponding condition number then characterizes the expected convergence rate of iterations. As a consequence, our results specify the convergence rate of the stochastic gradient descent algorithm, in terms of the mini-batch size n, when used for solving systems of linear equations.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/21/2017

A Novel Stochastic Stratified Average Gradient Method: Convergence Rate and Its Complexity

SGD (Stochastic Gradient Descent) is a popular algorithm for large scale...
research
01/24/2020

A Sharp Convergence Rate for the Asynchronous Stochastic Gradient Descent

We give a sharp convergence rate for the asynchronous stochastic gradien...
research
08/09/2015

A Linearly-Convergent Stochastic L-BFGS Algorithm

We propose a new stochastic L-BFGS algorithm and prove a linear converge...
research
02/15/2023

Randomized Orthogonal Projection Methods for Krylov Subspace Solvers

Randomized orthogonal projection methods (ROPMs) can be used to speed up...
research
09/19/2022

Strong convergence of parabolic rate 1 of discretisations of stochastic Allen-Cahn-type equations

Consider the approximation of stochastic Allen-Cahn-type equations (i.e....
research
08/22/2020

Distributed Linear Equations over Random Networks

Distributed linear algebraic equation over networks, where nodes hold a ...
research
02/21/2020

Debiasing Stochastic Gradient Descent to handle missing values

A major caveat of large scale data is their incom-pleteness. We propose ...

Please sign up or login with your details

Forgot password? Click here to reset