Adaptively Sketched Bregman Projection Methods for Linear Systems

12/29/2021
by   Ziyang Yuan, et al.
0

The sketch-and-project, as a general archetypal algorithm for solving linear systems, unifies a variety of randomized iterative methods such as the randomized Kaczmarz and randomized coordinate descent. However, since it aims to find a least-norm solution from a linear system, the randomized sparse Kaczmarz can not be included. This motivates us to propose a more general framework, called sketched Bregman projection (SBP) method, in which we are able to find solutions with certain structures from linear systems. To generalize the concept of adaptive sampling to the SBP method, we show how the progress, measured by Bregman distance, of single step depends directly on a sketched loss function. Theoretically, we provide detailed global convergence results for the SBP method with different adaptive sampling rules. At last, for the (sparse) Kaczmarz methods, a group of numerical simulations are tested, with which we verify that the methods utilizing sampling Kaczmarz-Motzkin rule demands the fewest computational costs to achieve a given error bound comparing to the corresponding methods with other sampling rules.

READ FULL TEXT
research
09/09/2019

Adaptive Sketch-and-Project Methods for Solving Linear Systems

We present new adaptive sampling rules for the sketch-and-project method...
research
10/10/2021

Selectable Set Randomized Kaczmarz

The Randomized Kaczmarz method (RK) is a stochastic iterative method for...
research
12/05/2020

Sketch Project Methods for Linear Feasibility Problems: Greedy Sampling Momentum

We develop two greedy sampling rules for the Sketch Project method f...
research
12/24/2020

Stochastic Steepest Descent Methods for Linear Systems: Greedy Sampling Momentum

Recently proposed adaptive Sketch Project (SP) methods connect sever...
research
10/11/2021

Randomized Extended Kaczmarz is a Limit Point of Sketch-and-Project

The sketch-and-project (SAP) framework for solving systems of linear equ...
research
06/12/2023

A Weighted Randomized Sparse Kaczmarz Method for Solving Linear Systems

The randomized sparse Kaczmarz method, designed for seeking the sparse s...
research
01/13/2021

Sparse Sampling Kaczmarz-Motzkin Method with Linear Convergence

The randomized sparse Kaczmarz method was recently proposed to recover s...

Please sign up or login with your details

Forgot password? Click here to reset