Robust regression with covariate filtering: Heavy tails and adversarial contamination

09/27/2020
by   Ankit Pensia, et al.
0

We study the problem of linear regression where both covariates and responses are potentially (i) heavy-tailed and (ii) adversarially contaminated. Several computationally efficient estimators have been proposed for the simpler setting where the covariates are sub-Gaussian and uncontaminated; however, these estimators may fail when the covariates are either heavy-tailed or contain outliers. In this work, we show how to modify the Huber regression, least trimmed squares, and least absolute deviation estimators to obtain estimators which are simultaneously computationally and statistically efficient in the stronger contamination model. Our approach is quite simple, and consists of applying a filtering algorithm to the covariates, and then applying the classical robust regression estimators to the remaining data. We show that the Huber regression estimator achieves near-optimal error rates in this setting, whereas the least trimmed squares and least absolute deviation estimators can be made to achieve near-optimal error after applying a postprocessing step.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/25/2021

Distribution-Free Robust Linear Regression

We study random design linear regression with no assumptions on the dist...
research
01/09/2023

On Consistency and Asymptotic Normality of Least Absolute Deviation Estimators for 2-dimensional Sinusoidal Model

Estimation of the parameters of a 2-dimensional sinusoidal model is a fu...
research
06/15/2022

Robust and Sparse Estimation of Linear Regression Coefficients with Heavy-tailed Noises and Covariates

Robust and sparse estimation of linear regression coefficients is invest...
research
07/01/2019

A Unified Approach to Robust Mean Estimation

In this paper, we develop connections between two seemingly disparate, b...
research
02/22/2021

Adversarial robust weighted Huber regression

We propose a novel method to estimate the coefficients of linear regress...
research
02/03/2021

Query Complexity of Least Absolute Deviation Regression via Robust Uniform Convergence

Consider a regression problem where the learner is given a large collect...
research
09/07/2021

Dynamic Network Regression

Network data are increasingly available in various research fields, moti...

Please sign up or login with your details

Forgot password? Click here to reset