Local stability and robustness of sparse dictionary learning in the presence of noise

10/02/2012
by   Rodolphe Jenatton, et al.
0

A popular approach within the signal processing and machine learning communities consists in modelling signals as sparse linear combinations of atoms selected from a learned dictionary. While this paradigm has led to numerous empirical successes in various fields ranging from image to audio processing, there have only been a few theoretical arguments supporting these evidences. In particular, sparse coding, or sparse dictionary learning, relies on a non-convex procedure whose local minima have not been fully analyzed yet. In this paper, we consider a probabilistic model of sparse signals, and show that, with high probability, sparse coding admits a local minimum around the reference dictionary generating the signals. Our study takes into account the case of over-complete dictionaries and noisy signals, thus extending previous work limited to noiseless settings and/or under-complete dictionaries. The analysis we conduct is non-asymptotic and makes it possible to understand how the key quantities of the problem, such as the coherence or the level of noise, can scale with respect to the dimension of the signals, the number of atoms, the sparsity and the number of observations.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/19/2014

Sparse and spurious: dictionary learning with noise and outliers

A popular approach within the signal processing and machine learning com...
research
05/28/2012

Learning Dictionaries with Bounded Self-Coherence

Sparse coding in learned dictionaries has been established as a successf...
research
05/17/2015

Local identifiability of l_1-minimization dictionary learning: a sufficient and almost necessary condition

We study the theoretical properties of learning a dictionary from N sign...
research
10/23/2010

Collaborative Sources Identification in Mixed Signals via Hierarchical Sparse Modeling

A collaborative framework for detecting the different sources in mixed s...
research
02/24/2023

Hiding Data Helps: On the Benefits of Masking for Sparse Coding

Sparse coding refers to modeling a signal as sparse linear combinations ...
research
09/21/2014

Analyzing sparse dictionaries for online learning with kernels

Many signal processing and machine learning methods share essentially th...
research
02/22/2019

Unique Sharp Local Minimum in ℓ_1-minimization Complete Dictionary Learning

We study the problem of globally recovering a dictionary from a set of s...

Please sign up or login with your details

Forgot password? Click here to reset