Robust regression with covariate filtering: Heavy tails and adversarial contamination

09/27/2020
by   Ankit Pensia, et al.
0

We study the problem of linear regression where both covariates and responses are potentially (i) heavy-tailed and (ii) adversarially contaminated. Several computationally efficient estimators have been proposed for the simpler setting where the covariates are sub-Gaussian and uncontaminated; however, these estimators may fail when the covariates are either heavy-tailed or contain outliers. In this work, we show how to modify the Huber regression, least trimmed squares, and least absolute deviation estimators to obtain estimators which are simultaneously computationally and statistically efficient in the stronger contamination model. Our approach is quite simple, and consists of applying a filtering algorithm to the covariates, and then applying the classical robust regression estimators to the remaining data. We show that the Huber regression estimator achieves near-optimal error rates in this setting, whereas the least trimmed squares and least absolute deviation estimators can be made to achieve near-optimal error after applying a postprocessing step.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

02/25/2021

Distribution-Free Robust Linear Regression

We study random design linear regression with no assumptions on the dist...
07/01/2019

A Unified Approach to Robust Mean Estimation

In this paper, we develop connections between two seemingly disparate, b...
02/22/2021

Adversarial robust weighted Huber regression

We propose a novel method to estimate the coefficients of linear regress...
02/03/2021

Query Complexity of Least Absolute Deviation Regression via Robust Uniform Convergence

Consider a regression problem where the learner is given a large collect...
06/29/2020

Robust Linear Regression: Optimal Rates in Polynomial Time

We obtain a robust and computationally efficient estimator for Linear Re...
07/06/2021

Distributed Adaptive Huber Regression

Distributed data naturally arise in scenarios involving multiple sources...
06/01/2019

Robust approximate linear regression without correspondence

Estimating regression coefficients using unordered multisets of covariat...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.