Are sketch-and-precondition least squares solvers numerically stable?

02/14/2023
by   Maike Meier, et al.
0

Sketch-and-precondition techniques are popular for solving large least squares (LS) problems of the form Ax=b with A∈ℝ^m× n and m≫ n. This is where A is “sketched" to a smaller matrix SA with S∈ℝ^⌈ cn⌉× m for some constant c>1 before an iterative LS solver computes the solution to Ax=b with a right preconditioner P, where P is constructed from SA. Popular sketch-and-precondition LS solvers are Blendenpik and LSRN. We show that the sketch-and-precondition technique is not numerically stable for ill-conditioned LS problems. Instead, we propose using an unpreconditioned iterative LS solver on (AP)y=b with x=Py when accuracy is a concern. Provided the condition number of A is smaller than the reciprocal of the unit round-off, we show that this modification ensures that the computed solution has a comparable backward error to the iterative LS solver applied to a well-conditioned matrix. Using smoothed analysis, we model floating-point rounding errors to provide a convincing argument that our modification is expected to compute a backward stable solution even for arbitrarily ill-conditioned LS problems.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/19/2020

A Stabilized GMRES Method for Solving Underdetermined Least Squares Problems

Consider using the right-preconditioned generalized minimal residual (AB...
research
08/04/2020

Well-Conditioned Methods for Ill-Conditioned Systems: Linear Regression with Semi-Random Noise

Classical iterative algorithms for linear system solving and regression ...
research
04/29/2021

Fast Convex Quadratic Optimization Solvers with Adaptive Sketching-based Preconditioners

We consider least-squares problems with quadratic regularization and pro...
research
03/28/2022

Acceleration of multiple precision solver for ill-conditioned algebraic equations with lower precision eigensolver

There are some types of ill-conditioned algebraic equations that have di...
research
12/07/2019

Regularized Momentum Iterative Hessian Sketch for Large Scale Linear System of Equations

In this article, Momentum Iterative Hessian Sketch (M-IHS) techniques, a...
research
03/14/2022

Randomized algorithms for Tikhonov regularization in linear least squares

We describe two algorithms to efficiently solve regularized linear least...
research
05/18/2022

Coarsest-level improvements in multigrid for lattice QCD on large-scale computers

Numerical simulations of quantum chromodynamics (QCD) on a lattice requi...

Please sign up or login with your details

Forgot password? Click here to reset