DeepAI AI Chat
Log In Sign Up

The greedy side of the LASSO: New algorithms for weighted sparse recovery via loss function-based orthogonal matching pursuit

03/01/2023
by   Sina Mohammad-Taheri, et al.
0

We propose a class of greedy algorithms for weighted sparse recovery by considering new loss function-based generalizations of Orthogonal Matching Pursuit (OMP). Given a (regularized) loss function, the proposed algorithms alternate the iterative construction of the signal support via greedy index selection and a signal update based on solving a local data-fitting problem restricted to the current support. We show that greedy selection rules associated with popular weighted sparsity-promoting loss functions admit explicitly computable and simple formulas. Specifically, we consider ℓ^0- and ℓ^1-based versions of the weighted LASSO (Least Absolute Shrinkage and Selection Operator), the Square-Root LASSO (SR-LASSO) and the Least Absolute Deviations LASSO (LAD-LASSO). Through numerical experiments on Gaussian compressive sensing and high-dimensional function approximation, we demonstrate the effectiveness of the proposed algorithms and empirically show that they inherit desirable characteristics from the corresponding loss functions, such as SR-LASSO's noise-blind optimal parameter tuning and LAD-LASSO's fault tolerance. In doing so, our study sheds new light on the connection between greedy sparse recovery and convex relaxation.

READ FULL TEXT

page 1

page 2

page 3

page 4

10/25/2018

Sparse approximation of multivariate functions from small datasets via weighted orthogonal matching pursuit

We show the potential of greedy recovery strategies for the sparse appro...
02/20/2013

Matching Pursuit LASSO Part II: Applications and Sparse Recovery over Batch Signals

Matching Pursuit LASSIn Part I TanPMLPart1, a Matching Pursuit LASSO (MP...
08/19/2020

Robust Instance-Optimal Recovery of Sparse Signals at Unknown Noise Levels

We consider the problem of sparse signal recovery from noisy measurement...
05/04/2018

Hedging parameter selection for basis pursuit

In Compressed Sensing and high dimensional estimation, signal recovery o...
02/25/2023

Average case analysis of Lasso under ultra-sparse conditions

We analyze the performance of the least absolute shrinkage and selection...
05/19/2018

Sequential adaptive elastic net approach for single-snapshot source localization

This paper proposes efficient algorithms for accurate recovery of direct...
07/14/2023

Performance of ℓ_1 Regularization for Sparse Convex Optimization

Despite widespread adoption in practice, guarantees for the LASSO and Gr...