Recovery and Generalization in Over-Realized Dictionary Learning

06/11/2020
by   Jeremias Sulam, et al.
0

In over two decades of research, the field of dictionary learning has gathered a large collection of successful applications, and theoretical guarantees for model recovery are known only whenever optimization is carried out in the same model class as that of the underlying dictionary. This work characterizes the surprising phenomenon that dictionary recovery can be facilitated by searching over the space of larger over-realized models. This observation is general and independent of the specific dictionary learning algorithm used. We thoroughly demonstrate this observation in practice and provide a theoretical analysis of this phenomenon by tying recovery measures to generalization bounds. We further show that an efficient and provably correct distillation mechanism can be employed to recover the correct atoms from the over-realized model. As a result, our meta-algorithm provides dictionary estimates with consistently better recovery of the ground-truth model.

READ FULL TEXT

page 8

page 16

page 17

research
02/28/2019

NOODL: Provable Online Dictionary Learning and Sparse Coding

We consider the dictionary learning problem, where the aim is to model t...
research
02/24/2023

Hiding Data Helps: On the Benefits of Masking for Sparse Coding

Sparse coding refers to modeling a signal as sparse linear combinations ...
research
04/05/2015

Efficient Dictionary Learning via Very Sparse Random Projections

Performing signal processing tasks on compressive measurements of data h...
research
10/23/2022

Simple Alternating Minimization Provably Solves Complete Dictionary Learning

This paper focuses on complete dictionary learning problem, where the go...
research
12/29/2018

Monocular 3D Pose Recovery via Nonconvex Sparsity with Theoretical Analysis

For recovering 3D object poses from 2D images, a prevalent method is to ...
research
06/06/2019

Complete Dictionary Learning via ℓ^4-Norm Maximization over the Orthogonal Group

This paper considers the fundamental problem of learning a complete (ort...
research
03/19/2016

Buried object detection using handheld WEMI with task-driven extended functions of multiple instances

Many effective supervised discriminative dictionary learning methods hav...

Please sign up or login with your details

Forgot password? Click here to reset