Support Recovery for Orthogonal Matching Pursuit: Upper and Lower bounds

06/27/2021
by   Raghav Somani, et al.
0

This paper studies the problem of sparse regression where the goal is to learn a sparse vector that best optimizes a given objective function. Under the assumption that the objective function satisfies restricted strong convexity (RSC), we analyze orthogonal matching pursuit (OMP), a greedy algorithm that is used heavily in applications, and obtain support recovery result as well as a tight generalization error bound for OMP. Furthermore, we obtain lower bounds for OMP, showing that both our results on support recovery and generalization error are tight up to logarithmic factors. To the best of our knowledge, these support recovery and generalization bounds are the first such matching upper and lower bounds (up to logarithmic factors) for {\em any} sparse regression algorithm under the RSC assumption.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/10/2018

A Sharp Condition for Exact Support Recovery of Sparse Signals With Orthogonal Matching Pursuit

Support recovery of sparse signals from noisy measurements with orthogon...
research
12/11/2020

A New Bound on Approximate Support Recovery

Orthogonal matching pursuit (OMP) is a greedy algorithm popularly being ...
research
12/24/2020

A Tight Lower Bound for Uniformly Stable Algorithms

Leveraging algorithmic stability to derive sharp generalization bounds i...
research
12/25/2013

A Convex Formulation for Mixed Regression with Two Components: Minimax Optimal Rates

We consider the mixed regression problem with two components, under adve...
research
07/15/2023

Sharp Convergence Rates for Matching Pursuit

We study the fundamental limits of matching pursuit, or the pure greedy ...
research
02/11/2016

On the Difficulty of Selecting Ising Models with Approximate Recovery

In this paper, we consider the problem of estimating the underlying grap...
research
01/29/2021

Lower Bounds on Information Requirements for Causal Network Inference

Recovery of the causal structure of dynamic networks from noisy measurem...

Please sign up or login with your details

Forgot password? Click here to reset