DeepAI AI Chat
Log In Sign Up

Convex vs nonconvex approaches for sparse estimation: GLasso, Multiple Kernel Learning and Hyperparameter GLasso

by   Aleksandr Y. Aravkin, et al.
Università di Padova
University of Washington

The popular Lasso approach for sparse estimation can be derived via marginalization of a joint density associated with a particular stochastic model. A different marginalization of the same probabilistic model leads to a different non-convex estimator where hyperparameters are optimized. Extending these arguments to problems where groups of variables have to be estimated, we study a computational scheme for sparse estimation that differs from the Group Lasso. Although the underlying optimization problem defining this estimator is non-convex, an initialization strategy based on a univariate Bayesian forward selection scheme is presented. This also allows us to define an effective non-convex estimator where only one scalar variable is involved in the optimization process. Theoretical arguments, independent of the correctness of the priors entering the sparse model, are included to clarify the advantages of this non-convex technique in comparison with other convex estimators. Numerical experiments are also used to compare the performance of these approaches.


page 1

page 2

page 3

page 4


A Convex-Nonconvex Strategy for Grouped Variable Selection

This paper deals with the grouped variable selection problem. A widely u...

Non-convex Global Minimization and False Discovery Rate Control for the TREX

The TREX is a recently introduced method for performing sparse high-dime...

A Unifying Framework of High-Dimensional Sparse Estimation with Difference-of-Convex (DC) Regularizations

Under the linear regression framework, we study the variable selection p...

Integrated Principal Components Analysis

Data integration, or the strategic analysis of multiple sources of data ...

An algorithm for non-convex off-the-grid sparse spike estimation with a minimum separation constraint

Theoretical results show that sparse off-the-grid spikes can be estimate...

Audio Dequantization Using (Co)Sparse (Non)Convex Methods

The paper deals with the hitherto neglected topic of audio dequantizatio...

Integrated Conditional Estimation-Optimization

Many real-world optimization problems involve uncertain parameters with ...