Sparse graph based sketching for fast numerical linear algebra

by   Dong Hu, et al.

In recent years, a variety of randomized constructions of sketching matrices have been devised, that have been used in fast algorithms for numerical linear algebra problems, such as least squares regression, low-rank approximation, and the approximation of leverage scores. A key property of sketching matrices is that of subspace embedding. In this paper, we study sketching matrices that are obtained from bipartite graphs that are sparse, i.e., have left degreeΒ s that is small. In particular, we explore two popular classes of sparse graphs, namely, expander graphs and magical graphs. For a given subspace π’°βŠ†β„^n of dimension k, we show that the magical graph with left degree s=2 yields a (1Β±Ο΅) β„“_2-subspace embedding for 𝒰, if the number of right vertices (the sketch size) m = π’ͺ(k^2/Ο΅^2). The expander graph with s = π’ͺ(log k/Ο΅) yields a subspace embedding for m = π’ͺ(k log k/Ο΅^2). We also discuss the construction of sparse sketching matrices with reduced randomness using expanders based on error-correcting codes. Empirical results on various synthetic and real datasets show that these sparse graph sketching matrices work very well in practice.



page 1

page 2

page 3

page 4

βˆ™ 02/04/2020

Randomized Numerical Linear Algebra: Foundations Algorithms

This survey describes probabilistic algorithms for linear algebra comput...
βˆ™ 04/24/2018

On the construction of sparse matrices from expander graphs

We revisit the asymptotic analysis of probabilistic construction of adja...
βˆ™ 03/10/2017

Joint Embedding of Graphs

Feature extraction and dimension reduction for networks is critical in a...
βˆ™ 05/25/2021

Hashing embeddings of optimal dimension, with applications to linear least squares

The aim of this paper is two-fold: firstly, to present subspace embeddin...
βˆ™ 07/20/2020

On Learned Sketches for Randomized Numerical Linear Algebra

We study "learning-based" sketching approaches for diverse tasks in nume...
βˆ™ 08/02/2019

Deterministic Completion of Rectangular Matrices Using Asymmetric Ramanujan Graphs

In this paper we study the matrix completion problem: Suppose X ∈R^n_r Γ—...
βˆ™ 11/09/2020

Quantum-Inspired Algorithms from Randomized Numerical Linear Algebra

We create classical (non-quantum) dynamic data structures supporting que...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.