Non-Asymptotic Bounds for the ℓ_∞ Estimator in Linear Regression with Uniform Noise

08/17/2021
by   Yufei Yi, et al.
0

The Chebyshev or ℓ_∞ estimator is an unconventional alternative to the ordinary least squares in solving linear regressions. It is defined as the minimizer of the ℓ_∞ objective function β̂ := min_βY - 𝐗β_∞. The asymptotic distribution of the Chebyshev estimator under fixed number of covariates were recently studied (Knight, 2020), yet finite sample guarantees and generalizations to high-dimensional settings remain open. In this paper, we develop non-asymptotic upper bounds on the estimation error β̂-β^*_2 for a Chebyshev estimator β̂, in a regression setting with uniformly distributed noise ε_i∼ U([-a,a]) where a is either known or unknown. With relatively mild assumptions on the (random) design matrix 𝐗, we can bound the error rate by C_p/n with high probability, for some constant C_p depending on the dimension p and the law of the design. Furthermore, we illustrate that there exist designs for which the Chebyshev estimator is (nearly) minimax optimal. In addition we show that "Chebyshev's LASSO" has advantages over the regular LASSO in high dimensional situations, provided that the noise is uniform. Specifically, we argue that it achieves a much faster rate of estimation under certain assumptions on the growth rate of the sparsity level and the ambient dimension with respect to the sample size.

READ FULL TEXT

page 1

page 2

page 3

page 4

05/26/2019

Nonregular and Minimax Estimation of Individualized Thresholds in High Dimension with Binary Responses

Given a large number of covariates Z, we consider the estimation of a hi...
03/18/2021

Robust-to-outliers square-root LASSO, simultaneous inference with a MOM approach

We consider the least-squares regression problem with unknown noise vari...
09/20/2019

Does SLOPE outperform bridge regression?

A recently proposed SLOPE estimator (arXiv:1407.3824) has been shown to ...
06/21/2019

Max-Affine Regression: Provable, Tractable, and Near-Optimal Statistical Estimation

Max-affine regression refers to a model where the unknown regression fun...
11/12/2021

Distributed Sparse Regression via Penalization

We study sparse linear regression over a network of agents, modeled as a...
06/20/2016

On the prediction loss of the lasso in the partially labeled setting

In this paper we revisit the risk bounds of the lasso estimator in the c...
06/16/2020

Risk bounds when learning infinitely many response functions by ordinary linear regression

Consider the problem of learning a large number of response functions si...