Statistical consistency and asymptotic normality for high-dimensional robust M-estimators

01/01/2015
by   Po-Ling Loh, et al.
0

We study theoretical properties of regularized robust M-estimators, applicable when data are drawn from a sparse high-dimensional linear model and contaminated by heavy-tailed distributions and/or outliers in the additive errors and covariates. We first establish a form of local statistical consistency for the penalized regression estimators under fairly mild conditions on the error distribution: When the derivative of the loss function is bounded and satisfies a local restricted curvature condition, all stationary points within a constant radius of the true regression vector converge at the minimax rate enjoyed by the Lasso with sub-Gaussian errors. When an appropriate nonconvex regularizer is used in place of an l_1-penalty, we show that such stationary points are in fact unique and equal to the local oracle solution with the correct support---hence, results on asymptotic normality in the low-dimensional case carry over immediately to the high-dimensional setting. This has important implications for the efficiency of regularized nonconvex M-estimators when the errors are heavy-tailed. Our analysis of the local curvature of the loss function also has useful consequences for optimization when the robust regression function and/or regularizer is nonconvex and the objective function possesses stationary points outside the local region. We show that as long as a composite gradient descent algorithm is initialized within a constant radius of the true regression vector, successive iterates will converge at a linear rate to a stationary point within the local region. Furthermore, the global optimum of a convex regularized robust regression function may be used to obtain a suitable initialization. The result is a novel two-step procedure that uses a convex M-estimator to achieve consistency and a nonconvex M-estimator to increase efficiency.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/10/2013

Regularized M-estimators with nonconvexity: Statistical and algorithmic theory for local optima

We provide novel theoretical results regarding local optima of regulariz...
research
07/09/2019

Nonconvex Regularized Robust Regression with Oracle Properties in Polynomial Time

This paper investigates tradeoffs among optimization errors, statistical...
research
03/11/2015

Optimal prediction for sparse linear models? Lower bounds for coordinate-separable M-estimators

For the problem of high-dimensional sparse linear regression, it is know...
research
08/18/2019

Semiparametric Expectile Regression for High-dimensional Heavy-tailed and Heterogeneous Data

Recently, high-dimensional heterogeneous data have attracted a lot of at...
research
11/19/2019

Sparse recovery via nonconvex regularized M-estimators over ℓ_q-balls

In this paper, we analyse the recovery properties of nonconvex regulariz...
research
11/03/2020

Support estimation in high-dimensional heteroscedastic mean regression

A current strand of research in high-dimensional statistics deals with r...
research
11/27/2018

High-dimensional Index Volatility Models via Stein's Identity

In this paper, we consider estimating the parametric components of index...

Please sign up or login with your details

Forgot password? Click here to reset