On Learned Sketches for Randomized Numerical Linear Algebra

by   Simin Liu, et al.

We study "learning-based" sketching approaches for diverse tasks in numerical linear algebra: least-squares regression, ℓ_p regression, Huber regression, low-rank approximation (LRA), and k-means clustering. Sketching methods are used to quickly and approximately compute properties of large matrices. Linear maps called "sketches" are applied to compress data, and these concise representations are used to compute the desired properties. Specifically, we consider sparse sketches (such as CountSketch). Recent works have dealt with optimizing sketches for data distributions to perform better than their random counterparts. We extend this theme to several important and ubiquitous tasks, each of which requires a new analysis and novel practical methods. Specifically, our contributions are: 1) For all tasks, we introduce fast algorithms using learned sketches with worst-case guarantees. We give a simple task-agnostic method for retaining the worst-case guarantees of randomized sketching, which yields time-optimal algorithms for LRA and least-squares regression. Also, for k-means clustering, we give a faster alternative for retaining worst-case guarantees. 2) We show empirically that learned sketches are reliable in improving approximation accuracy, with comparison against "non-learned" sketching baselines. 3) We introduce a greedy algorithm for optimizing the location of the nonzero entries of a sparse sketch and prove guarantees for certain distributions on the LRA task. Previous work only looked at optimizing the values rather than the locations. Also, we show empirically that it further improves learned sketch performance.



There are no comments yet.


page 1

page 2

page 3

page 4


Learning-Based Low-Rank Approximations

We introduce a "learning-based" algorithm for the low-rank decomposition...

Sparse graph based sketching for fast numerical linear algebra

In recent years, a variety of randomized constructions of sketching matr...

Determinantal Point Processes in Randomized Numerical Linear Algebra

Randomized Numerical Linear Algebra (RandNLA) uses randomness to develop...

Faster Algorithms and Constant Lower Bounds for the Worst-Case Expected Error

The study of statistical estimation without distributional assumptions o...

Low Rank Approximation of a Matrix at Sub-linear Cost

A matrix algorithm performs at sub-linear cost if it uses much fewer flo...

Refinement of Low Rank Approximation of a Matrix at Sub-linear Cost

Low rank approximation (LRA) of a matrix is a hot subject of modern comp...

Low Rank Approximation at Sublinear Cost by Means of Subspace Sampling

Low Rank Approximation (LRA) of a matrix is a hot research subject, fund...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.