
Dual Extrapolation for Faster Lasso Solvers
Convex sparsityinducing regularizations are ubiquitous in highdimensio...
02/21/2018 ∙ by Massias Mathurin, et al. ∙ 0 ∙ shareread it

GAP Safe screening rules for sparse multitask and multiclass models
High dimensional regression benefits from sparsity promoting regularizat...
06/11/2015 ∙ by Eugene Ndiaye, et al. ∙ 0 ∙ shareread it

From safe screening rules to working sets for faster Lassotype solvers
Convex sparsitypromoting regularizations are ubiquitous in modern stati...
03/21/2017 ∙ by Mathurin Massias, et al. ∙ 0 ∙ shareread it

ExSIS: Extended Sure Independence Screening for Ultrahighdimensional Linear Models
Statistical inference can be computationally prohibitive in ultrahighdi...
08/21/2017 ∙ by Talal Ahmed, et al. ∙ 0 ∙ shareread it

Distributed Coordinate Descent for Generalized Linear Models with Regularization
Generalized linear model with L_1 and L_2 regularization is a widely use...
11/07/2016 ∙ by Ilya Trofimov, et al. ∙ 0 ∙ shareread it

Screening Rules for Lasso with NonConvex Sparse Regularizers
Leveraging on the convexity of the Lasso problem , screening rules help ...
02/16/2019 ∙ by Alain Rakotomamonjy, et al. ∙ 18 ∙ shareread it

The Symmetry of a Simple Optimization Problem in Lasso Screening
Recently dictionary screening has been proposed as an effective way to i...
08/21/2016 ∙ by Yun Wang, et al. ∙ 0 ∙ shareread it
Dual Extrapolation for Sparse Generalized Linear Models
Generalized Linear Models (GLM) form a wide class of regression and classification models, where prediction is a function of a linear combination of the input variables. For statistical inference in high dimension, sparsity inducing regularizations have proven to be useful while offering statistical guarantees. However, solving the resulting optimization problems can be challenging: even for popular iterative algorithms such as coordinate descent, one needs to loop over a large number of variables. To mitigate this, techniques known as screening rules and working sets diminish the size of the optimization problem at hand, either by progressively removing variables, or by solving a growing sequence of smaller problems. For both techniques, significant variables are identified thanks to convex duality arguments. In this paper, we show that the dual iterates of a GLM exhibit a Vector AutoRegressive (VAR) behavior after sign identification, when the primal problem is solved with proximal gradient descent or cyclic coordinate descent. Exploiting this regularity, one can construct dual points that offer tighter certificates of optimality, enhancing the performance of screening rules and helping to design competitive working set algorithms.
READ FULL TEXT