A spectral algorithm for robust regression with subgaussian rates

by   Jules Depersin, et al.

We study a new linear up to quadratic time algorithm for linear regression in the absence of strong assumptions on the underlying distributions of samples, and in the presence of outliers. The goal is to design a procedure which comes with actual working code that attains the optimal sub-gaussian error bound even though the data have only finite moments (up to L_4) and in the presence of possibly adversarial outliers. A polynomial-time solution to this problem has been recently discovered but has high runtime due to its use of Sum-of-Square hierarchy programming. At the core of our algorithm is an adaptation of the spectral method introduced for the mean estimation problem to the linear regression problem. As a by-product we established a connection between the linear regression problem and the furthest hyperplane problem. From a stochastic point of view, in addition to the study of the classical quadratic and multiplier processes we introduce a third empirical process that comes naturally in the study of the statistical properties of the algorithm.



page 1

page 2

page 3

page 4


Algorithms for Heavy-Tailed Statistics: Regression, Covariance Estimation, and Beyond

We study efficient algorithms for linear regression and covariance estim...

Fast Mean Estimation with Sub-Gaussian Rates

We propose an estimator for the mean of a random vector in R^d that can ...

A Fast Spectral Algorithm for Mean Estimation with Sub-Gaussian Rates

We study the algorithmic problem of estimating the mean of heavy-tailed ...

An Algebraic-Geometric Approach to Shuffled Linear Regression

Shuffled linear regression is the problem of performing a linear regress...

Link Prediction in the Stochastic Block Model with Outliers

The Stochastic Block Model is a popular model for network analysis in th...

Robust Meta-learning for Mixed Linear Regression with Small Batches

A common challenge faced in practical supervised learning, such as medic...

Robust Generalized Method of Moments: A Finite Sample Viewpoint

For many inference problems in statistics and econometrics, the unknown ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.