Safe Active Feature Selection for Sparse Learning

06/15/2018
by   Shaogang Ren, et al.
0

We present safe active incremental feature selection (SAIF) to scale up the computation of LASSO solutions. SAIF does not require a solution from a heavier penalty parameter as in sequential screening or updating the full model for each iteration as in dynamic screening. Different from these existing screening methods, SAIF starts from a small number of features and incrementally recruits active features and updates the significantly reduced model. Hence, it is much more computationally efficient and scalable with the number of features. More critically, SAIF has the safe guarantee as it has the convergence guarantee to the optimal solution to the original full LASSO problem. Such an incremental procedure and theoretical convergence guarantee can be extended to fused LASSO problems. Compared with state-of-the-art screening methods as well as working set and homotopy methods, which may not always guarantee the optimal solution, SAIF can achieve superior or comparable efficiency and high scalability with the safe guarantee when facing extremely high dimensional data sets. Experiments with both synthetic and real-world data sets show that SAIF can be up to 50 times faster than dynamic screening, and hundreds of times faster than computing LASSO or fused LASSO solutions without screening.

READ FULL TEXT
research
04/27/2017

Efficient Feature Screening for Lasso-Type Problems via Hybrid Safe-Strong Rules

The lasso model has been widely used for model selection in data mining,...
research
07/29/2013

Safe Screening With Variational Inequalities and Its Application to LASSO

Sparse learning techniques have been routinely used for feature selectio...
research
10/10/2018

ET-Lasso: Efficient Tuning of Lasso for High-Dimensional Data

The L1 regularization (Lasso) has proven to be a versatile tool to selec...
research
07/01/2023

Safe Screening for Unbalanced Optimal Transport

This paper introduces a framework that utilizes the Safe Screening techn...
research
02/22/2020

Safe Screening for the Generalized Conditional Gradient Method

The conditional gradient method (CGM) has been widely used for fast spar...
research
10/14/2018

Convex Hull Approximation of Nearly Optimal Lasso Solutions

In an ordinary feature selection procedure, a set of important features ...
research
07/08/2020

Accelerated Sparse Bayesian Learning via Screening Test and Its Applications

In high-dimensional settings, sparse structures are critical for efficie...

Please sign up or login with your details

Forgot password? Click here to reset