Does SLOPE outperform bridge regression?

09/20/2019
by   Shuaiwen Wang, et al.
0

A recently proposed SLOPE estimator (arXiv:1407.3824) has been shown to adaptively achieve the minimax ℓ_2 estimation rate under high-dimensional sparse linear regression models (arXiv:1503.08393). Such minimax optimality holds in the regime where the sparsity level k, sample size n, and dimension p satisfy k/p → 0, k p/n → 0. In this paper, we characterize the estimation error of SLOPE under the complementary regime where both k and n scale linearly with p, and provide new insights into the performance of SLOPE estimators. We first derive a concentration inequality for the finite sample mean square error (MSE) of SLOPE. The quantity that MSE concentrates around takes a complicated and implicit form. With delicate analysis of the quantity, we prove that among all SLOPE estimators, LASSO is optimal for estimating k-sparse parameter vectors that do not have tied non-zero components in the low noise scenario. On the other hand, in the large noise scenario, the family of SLOPE estimators are sub-optimal compared with bridge regression such as the Ridge estimator.

READ FULL TEXT

Authors

page 1

page 2

page 3

page 4

08/17/2021

Non-Asymptotic Bounds for the ℓ_∞ Estimator in Linear Regression with Uniform Noise

The Chebyshev or ℓ_∞ estimator is an unconventional alternative to the o...
11/05/2021

Liu Estimator in the Multinomial Logistic Regression Model

This paper considers the Liu estimator in the multinomial logistic regre...
12/27/2019

Minimax Semiparametric Learning With Approximate Sparsity

Many objects of interest can be expressed as a linear, mean square conti...
12/25/2018

Optimal False Discovery Control of Minimax Estimator

In the analysis of high dimensional regression models, there are two imp...
07/07/2020

Estimation and Inference with Trees and Forests in High Dimensions

We analyze the finite sample mean squared error (MSE) performance of reg...
07/08/2020

A new generalized newsvendor model with random demand

Newsvendor problem is an extensively researched topic in inventory manag...
06/19/2020

Information theoretic limits of learning a sparse rule

We consider generalized linear models in regimes where the number of non...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.