The Strong Screening Rule for SLOPE

05/07/2020
by   Johan Larsson, et al.
0

Extracting relevant features from data sets where the number of observations (n) is much smaller then the number of predictors (p) is a major challenge in modern statistics. Sorted L-One Penalized Estimation (SLOPE), a generalization of the lasso, is a promising method within this setting. Current numerical procedures for SLOPE, however, lack the efficiency that respective tools for the lasso enjoy, particularly in the context of estimating a complete regularization path. A key component in the efficiency of the lasso is predictor screening rules: rules that allow predictors to be discarded before estimating the model. This is the first paper to establish such a rule for SLOPE. We develop a screening rule for SLOPE by examining its subdifferential and show that this rule is a generalization of the strong rule for the lasso. Our rule is heuristic, which means that it may discard predictors erroneously. We present conditions under which this may happen and show that such situations are rare and easily safeguarded against by a simple check of the optimality conditions. Our numerical experiments show that the rule performs well in practice, leading to improvements by orders of magnitude for data in the p ≫ n domain, as well as incurring no additional computational overhead when n ≫ p. We also examine the effect of correlation structures in the design matrix on the rule and discuss algorithmic strategies for employing the rule. Finally, we provide an efficient implementation of the rule in our R package SLOPE.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/27/2021

The Hessian Screening Rule

Predictor screening rules, which discard predictors from the design matr...
research
11/16/2012

Lasso Screening Rules via Dual Polytope Projection

Lasso is a widely used regression technique to find sparse representatio...
research
05/12/2021

Look-Ahead Screening Rules for the Lasso

The lasso is a popular method to induce shrinkage and sparsity in the so...
research
11/09/2010

Strong rules for discarding predictors in lasso-type problems

We consider rules for discarding predictors in lasso regression and rela...
research
08/02/2022

A Screening Strategy for Structured Optimization Involving Nonconvex ℓ_q,p Regularization

In this paper, we develop a simple yet effective screening rule strategy...
research
12/17/2020

l1-norm quantile regression screening rule via the dual circumscribed sphere

l1-norm quantile regression is a common choice if there exists outlier o...
research
05/13/2014

Efficient Implementations of the Generalized Lasso Dual Path Algorithm

We consider efficient implementations of the generalized lasso dual path...

Please sign up or login with your details

Forgot password? Click here to reset