Target-Embedding Autoencoders for Supervised Representation Learning

01/23/2020
by   Daniel Jarrett, et al.
0

Autoencoder-based learning has emerged as a staple for disciplining representations in unsupervised and semi-supervised settings. This paper analyzes a framework for improving generalization in a purely supervised setting, where the target space is high-dimensional. We motivate and formalize the general framework of target-embedding autoencoders (TEA) for supervised prediction, learning intermediate latent representations jointly optimized to be both predictable from features as well as predictive of targets—encoding the prior that variations in targets are driven by a compact set of underlying factors. As our theoretical contribution, we provide a guarantee of generalization for linear TEAs by demonstrating uniform stability, interpreting the benefit of the auxiliary reconstruction task as a form of regularization. As our empirical contribution, we extend validation of this approach beyond existing static classification applications to multivariate sequence forecasting, verifying their advantage on both linear and nonlinear recurrent architectures—thereby underscoring the further generality of this framework beyond feedforward instantiations.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/04/2019

Generalization Bounds For Unsupervised and Semi-Supervised Learning With Autoencoders

Autoencoders are widely used for unsupervised learning and as a regulari...
research
10/21/2021

On the Regularization of Autoencoders

While much work has been devoted to understanding the implicit (and expl...
research
09/08/2022

Exploring Target Representations for Masked Autoencoders

Masked autoencoders have become popular training paradigms for self-supe...
research
09/26/2018

Unsupervised Adversarial Invariance

Data representations that contain all the information about target varia...
research
06/28/2021

Improving Prediction of Low-Prior Clinical Events with Simultaneous General Patient-State Representation Learning

Low-prior targets are common among many important clinical events, which...
research
09/20/2020

Provable Finite Data Generalization with Group Autoencoder

Deep Autoencoders (AEs) provide a versatile framework to learn a compres...

Please sign up or login with your details

Forgot password? Click here to reset