DeepAI AI Chat
Log In Sign Up

On Optimality Conditions for Auto-Encoder Signal Recovery

by   Devansh Arpit, et al.
University at Buffalo

Auto-Encoders are unsupervised models that aim to learn patterns from observed data by minimizing a reconstruction cost. The useful representations learned are often found to be sparse and distributed. On the other hand, compressed sensing and sparse coding assume a data generating process, where the observed data is generated from some true latent signal source, and try to recover the corresponding signal from measurements. Looking at auto-encoders from this signal recovery perspective enables us to have a more coherent view of these techniques. In this paper, in particular, we show that the true hidden representation can be approximately recovered if the weight matrices are highly incoherent with unit ℓ^2 row length and the bias vectors takes the value (approximately) equal to the negative of the data mean. The recovery also becomes more and more accurate as the sparsity in hidden signals increases. Additionally, we empirically demonstrate that auto-encoders are capable of recovering the data generating dictionary when only data samples are given.


page 1

page 2

page 3

page 4


Robust Compressed Sensing and Sparse Coding with the Difference Map

In compressed sensing, we wish to reconstruct a sparse signal x from obs...

Scalable Convolutional Dictionary Learning with Constrained Recurrent Sparse Auto-encoders

Given a convolutional dictionary underlying a set of observed signals, c...

Sparse Non-Negative Recovery from Biased Subgaussian Measurements using NNLS

We investigate non-negative least squares (NNLS) for the recovery of spa...

Using auto-encoders for solving ill-posed linear inverse problems

Compressed sensing algorithms recover a signal from its under-determined...

Joint Design of Measurement Matrix and Sparse Support Recovery Method via Deep Auto-encoder

Sparse support recovery arises in many applications in communications an...

Reconstruction of Hidden Representation for Robust Feature Extraction

This paper aims to develop a new and robust approach to feature represen...

Sparse Choice Models

Choice models, which capture popular preferences over objects of interes...