DeepAI AI Chat
Log In Sign Up

The greedy side of the LASSO: New algorithms for weighted sparse recovery via loss function-based orthogonal matching pursuit

by   Sina Mohammad-Taheri, et al.

We propose a class of greedy algorithms for weighted sparse recovery by considering new loss function-based generalizations of Orthogonal Matching Pursuit (OMP). Given a (regularized) loss function, the proposed algorithms alternate the iterative construction of the signal support via greedy index selection and a signal update based on solving a local data-fitting problem restricted to the current support. We show that greedy selection rules associated with popular weighted sparsity-promoting loss functions admit explicitly computable and simple formulas. Specifically, we consider ℓ^0- and ℓ^1-based versions of the weighted LASSO (Least Absolute Shrinkage and Selection Operator), the Square-Root LASSO (SR-LASSO) and the Least Absolute Deviations LASSO (LAD-LASSO). Through numerical experiments on Gaussian compressive sensing and high-dimensional function approximation, we demonstrate the effectiveness of the proposed algorithms and empirically show that they inherit desirable characteristics from the corresponding loss functions, such as SR-LASSO's noise-blind optimal parameter tuning and LAD-LASSO's fault tolerance. In doing so, our study sheds new light on the connection between greedy sparse recovery and convex relaxation.


page 1

page 2

page 3

page 4


Sparse approximation of multivariate functions from small datasets via weighted orthogonal matching pursuit

We show the potential of greedy recovery strategies for the sparse appro...

Matching Pursuit LASSO Part II: Applications and Sparse Recovery over Batch Signals

Matching Pursuit LASSIn Part I TanPMLPart1, a Matching Pursuit LASSO (MP...

Robust Instance-Optimal Recovery of Sparse Signals at Unknown Noise Levels

We consider the problem of sparse signal recovery from noisy measurement...

Hedging parameter selection for basis pursuit

In Compressed Sensing and high dimensional estimation, signal recovery o...

Average case analysis of Lasso under ultra-sparse conditions

We analyze the performance of the least absolute shrinkage and selection...

Sequential adaptive elastic net approach for single-snapshot source localization

This paper proposes efficient algorithms for accurate recovery of direct...

Performance of ℓ_1 Regularization for Sparse Convex Optimization

Despite widespread adoption in practice, guarantees for the LASSO and Gr...