Adaptive Hard Thresholding for Near-optimal Consistent Robust Regression

03/19/2019
by   Arun Sai Suggala, et al.
0

We study the problem of robust linear regression with response variable corruptions. We consider the oblivious adversary model, where the adversary corrupts a fraction of the responses in complete ignorance of the data. We provide a nearly linear time estimator which consistently estimates the true regression vector, even with 1-o(1) fraction of corruptions. Existing results in this setting either don't guarantee consistent estimates or can only handle a small fraction of corruptions. We also extend our estimator to robust sparse linear regression and show that similar guarantees hold in this setting. Finally, we apply our estimator to the problem of linear regression with heavy-tailed noise and show that our estimator consistently estimates the regression vector even when the noise has unbounded variance (e.g., Cauchy distribution), for which most existing results don't even apply. Our estimator is based on a novel variant of outlier removal via hard thresholding in which the threshold is chosen adaptively and crucially relies on randomness to escape bad fixed points of the non-convex hard thresholding operation.

READ FULL TEXT
research
09/30/2020

Regress Consistently when Oblivious Outliers Overwhelm

We give a novel analysis of the Huber loss estimator for consistent robu...
research
10/30/2022

Robust and Tuning-Free Sparse Linear Regression via Square-Root Slope

We consider the high-dimensional linear regression model and assume that...
research
07/01/2016

Efficient and Consistent Robust Time Series Analysis

We study the problem of robust time series analysis under the standard a...
research
08/24/2022

Outlier Robust and Sparse Estimation of Linear Regression Coefficients

We consider outlier robust and sparse estimation of linear regression co...
research
06/13/2019

Variance Estimation For Online Regression via Spectrum Thresholding

We consider the online linear regression problem, where the predictor ve...
research
06/08/2015

Robust Regression via Hard Thresholding

We study the problem of Robust Least Squares Regression (RLSR) where sev...

Please sign up or login with your details

Forgot password? Click here to reset