Fast Regression for Structured Inputs

03/14/2022
โˆ™
by   Raphael A. Meyer, et al.
โˆ™
0
โˆ™

We study the โ„“_p regression problem, which requires finding ๐ฑโˆˆโ„^d that minimizes ๐€๐ฑ-๐›_p for a matrix ๐€โˆˆโ„^n ร— d and response vector ๐›โˆˆโ„^n. There has been recent interest in developing subsampling methods for this problem that can outperform standard techniques when n is very large. However, all known subsampling approaches have run time that depends exponentially on p, typically, d^๐’ช(p), which can be prohibitively expensive. We improve on this work by showing that for a large class of common structured matrices, such as combinations of low-rank matrices, sparse matrices, and Vandermonde matrices, there are subsampling based methods for โ„“_p regression that depend polynomially on p. For example, we give an algorithm for โ„“_p regression on Vandermonde matrices that runs in time ๐’ช(nlog^3 n+(dp^2)^0.5+ฯ‰ยทpolylog n), where ฯ‰ is the exponent of matrix multiplication. The polynomial dependence on p crucially allows our algorithms to extend naturally to efficient algorithms for โ„“_โˆž regression, via approximation of โ„“_โˆž by โ„“_๐’ช(log n). Of practical interest, we also develop a new subsampling algorithm for โ„“_p regression for arbitrary matrices, which is simpler than previous approaches for p โ‰ฅ 4.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
โˆ™ 09/23/2021

Sparse Regression Faster than d^ฯ‰

The current complexity of regression is nearly linear in the complexity ...
research
โˆ™ 11/09/2020

Reduced-Rank Regression with Operator Norm Error

A common data analysis task is the reduced-rank regression problem: ...
research
โˆ™ 02/01/2023

A Nearly-Optimal Bound for Fast Regression with โ„“_โˆž Guarantee

Given a matrix Aโˆˆโ„^nร— d and a vector bโˆˆโ„^n, we consider the regression p...
research
โˆ™ 02/03/2016

Computing with quasiseparable matrices

The class of quasiseparable matrices is defined by a pair of bounds, cal...
research
โˆ™ 08/08/2019

On Symbolic Approaches for Computing the Matrix Permanent

Counting the number of perfect matchings in bipartite graphs, or equival...
research
โˆ™ 10/27/2022

Faster Linear Algebra for Distance Matrices

The distance matrix of a dataset X of n points with respect to a distanc...
research
โˆ™ 07/19/2012

The Fast Cauchy Transform and Faster Robust Linear Regression

We provide fast algorithms for overconstrained โ„“_p regression and relate...

Please sign up or login with your details

Forgot password? Click here to reset