Classification and regression using an outer approximation projection-gradient method

06/06/2015
by   Michel Barlaud, et al.
0

This paper deals with sparse feature selection and grouping for classification and regression. The classification or regression problems under consideration consists in minimizing a convex empirical risk function subject to an ℓ^1 constraint, a pairwise ℓ^∞ constraint, or a pairwise ℓ^1 constraint. Existing work, such as the Lasso formulation, has focused mainly on Lagrangian penalty approximations, which often require ad hoc or computationally expensive procedures to determine the penalization parameter. We depart from this approach and address the constrained problem directly via a splitting method. The structure of the method is that of the classical gradient-projection algorithm, which alternates a gradient step on the objective and a projection step onto the lower level set modeling the constraint. The novelty of our approach is that the projection step is implemented via an outer approximation scheme in which the constraint set is approximated by a sequence of simple convex sets consisting of the intersection of two half-spaces. Convergence of the iterates generated by the algorithm is established for a general smooth convex minimization problem with inequality constraints. Experiments on both synthetic and biological data show that our method outperforms penalty methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/09/2018

Frank-Wolfe Splitting via Augmented Lagrangian Method

Minimizing a function over an intersection of convex sets is an importan...
research
11/08/2017

Clustering with feature selection using alternating minimization, Application to computational biology

This paper deals with unsupervised clustering with feature selection. Th...
research
06/21/2021

Decentralized Constrained Optimization: Double Averaging and Gradient Projection

In this paper, we consider the convex, finite-sum minimization problem w...
research
09/12/2023

Fast Sparse PCA via Positive Semidefinite Projection for Unsupervised Feature Selection

In the field of unsupervised feature selection, sparse principal compone...
research
02/05/2019

Robust supervised classification and feature selection using a primal-dual method

This paper deals with supervised classification and feature selection in...
research
10/25/2022

Faster Projection-Free Augmented Lagrangian Methods via Weak Proximal Oracle

This paper considers a convex composite optimization problem with affine...
research
02/22/2020

Safe Screening for the Generalized Conditional Gradient Method

The conditional gradient method (CGM) has been widely used for fast spar...

Please sign up or login with your details

Forgot password? Click here to reset