Dual Extrapolation for Faster Lasso Solvers

02/21/2018
by   Massias Mathurin, et al.
0

Convex sparsity-inducing regularizations are ubiquitous in high-dimension machine learning, but their non-differentiability requires the use of iterative solvers. To accelerate such solvers, state-of-the-art approaches consist in reducing the size of the optimization problem at hand. In the context of regression, this can be achieved either by discarding irrelevant features (screening techniques) or by prioritizing features likely to be included in the support of the solution (working set techniques). Duality comes into play at several steps in these techniques. Here, we propose an extrapolation technique starting from a sequence of iterates in the dual that leads to the construction of an improved dual point. This enables a tighter control of optimality as used in stopping criterion, as well as better screening performance of Gap Safe rules. Finally, we propose a working set strategy based on an aggressive use of Gap Safe rules and our new dual point construction, which improves state-of-the-art time performance on Lasso problems.

READ FULL TEXT
research
03/21/2017

From safe screening rules to working sets for faster Lasso-type solvers

Convex sparsity-promoting regularizations are ubiquitous in modern stati...
research
05/13/2015

Mind the duality gap: safer rules for the Lasso

Screening rules allow to early discard irrelevant variables from the opt...
research
07/12/2019

Dual Extrapolation for Sparse Generalized Linear Models

Generalized Linear Models (GLM) form a wide class of regression and clas...
research
03/02/2022

Beyond GAP screening for Lasso by exploiting new dual cutting half-spaces with supplementary material

In this paper, we propose a novel safe screening test for Lasso. Our pro...
research
01/27/2020

On Newton Screening

Screening and working set techniques are important approaches to reducin...
research
07/20/2018

A Fast, Principled Working Set Algorithm for Exploiting Piecewise Linear Structure in Convex Problems

By reducing optimization to a sequence of smaller subproblems, working s...
research
02/22/2021

Expanding boundaries of Gap Safe screening

Sparse optimization problems are ubiquitous in many fields such as stati...

Please sign up or login with your details

Forgot password? Click here to reset