Generalization Bounds For Unsupervised and Semi-Supervised Learning With Autoencoders

02/04/2019
by   Baruch Epstein, et al.
0

Autoencoders are widely used for unsupervised learning and as a regularization scheme in semi-supervised learning. However, theoretical understanding of their generalization properties and of the manner in which they can assist supervised learning has been lacking. We utilize recent advances in the theory of deep learning generalization, together with a novel reconstruction loss, to provide generalization bounds for autoencoders. To the best of our knowledge, this is the first such bound. We further show that, under appropriate assumptions, an autoencoder with good generalization properties can improve any semi-supervised learning scheme. We support our theoretical results with empirical demonstrations.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/28/2019

A Review of Semi Supervised Learning Theories and Recent Advances

Semi-supervised learning, which has emerged from the beginning of this c...
research
01/23/2020

Target-Embedding Autoencoders for Supervised Representation Learning

Autoencoder-based learning has emerged as a staple for disciplining repr...
research
09/06/2018

Wasserstein Soft Label Propagation on Hypergraphs: Algorithm and Generalization Error Bounds

Inspired by recent interests of developing machine learning and data min...
research
09/12/2019

Generating Accurate Pseudo-labels via Hermite Polynomials for SSL Confidently

Rectified Linear Units (ReLUs) are among the most widely used activation...
research
06/27/2012

A Simple Algorithm for Semi-supervised Learning with Improved Generalization Error Bound

In this work, we develop a simple algorithm for semi-supervised regressi...
research
10/03/2021

Information-Theoretic Generalization Bounds for Iterative Semi-Supervised Learning

We consider iterative semi-supervised learning (SSL) algorithms that ite...
research
04/30/2015

Lateral Connections in Denoising Autoencoders Support Supervised Learning

We show how a deep denoising autoencoder with lateral connections can be...

Please sign up or login with your details

Forgot password? Click here to reset