What is a Denoising Autoencoder?
Denoising autoencoders are a stochastic version of standard autoencoders that reduces the risk of learning the identity function. Autoencoders are a class of neural networks used for feature selection and extraction, also called dimensionality reduction. In general, the more hidden layers in an autoencoder, the more refined this dimensional reduction can be. However, if an autoencoder has more hidden layers than inputs there is a risk the algorithm only learns the identity function during training, the point where the output simply equals the input, and then becomes useless.
Denoising autoencoders attempt to get around this risk of identity-function affiliation by introducing noise, i.e. randomly corrupting input so that the autoencoder must then “denoise” or reconstruct the original input.
Similar Types of Autoencoders:
- Sparse autoencoder – Includes more hidden nodes than inputs, but only a small amount are allowed to be active at once.
- Variational autoencoder – Uses the variational approach for learning latent representation, which creates an additional loss component and a new training algorithm called Stochastic Gradient Variational Bayes.
- Contractive autoencoder – Adds a regularizer in the objective function that forces the model to learn a function that is more flexible to variations in input values.
