Generalization bounds for deep thresholding networks

10/29/2020
by   Arash Behboodi, et al.
0

We consider compressive sensing in the scenario where the sparsity basis (dictionary) is not known in advance, but needs to be learned from examples. Motivated by the well-known iterative soft thresholding algorithm for the reconstruction, we define deep networks parametrized by the dictionary, which we call deep thresholding networks. Based on training samples, we aim at learning the optimal sparsifying dictionary and thereby the optimal network that reconstructs signals from their low-dimensional linear measurements. The dictionary learning is performed via minimizing the empirical risk. We derive generalization bounds by analyzing the Rademacher complexity of hypothesis classes consisting of such deep networks. We obtain estimates of the sample complexity that depend only linearly on the dimensions and on the depth.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/08/2021

Generalization Error Bounds for Iterative Recovery Algorithms Unfolded as Neural Networks

Motivated by the learned iterative soft thresholding algorithm (LISTA), ...
research
03/20/2014

On The Sample Complexity of Sparse Dictionary Learning

In the synthesis model signals are represented as a sparse combinations ...
research
12/25/2017

Overcomplete Frame Thresholding for Acoustic Scene Analysis

In this work, we derive a generic overcomplete frame thresholding scheme...
research
07/18/2017

Multiscale Residual Mixture of PCA: Dynamic Dictionaries for Optimal Basis Learning

In this paper we are interested in the problem of learning an over-compl...
research
02/09/2014

Dictionary learning for fast classification based on soft-thresholding

Classifiers based on sparse representations have recently been shown to ...
research
09/12/2023

Optimization Guarantees of Unfolded ISTA and ADMM Networks With Smooth Soft-Thresholding

Solving linear inverse problems plays a crucial role in numerous applica...

Please sign up or login with your details

Forgot password? Click here to reset