Controlling the Interaction Between Generation and Inference in Semi-Supervised Variational Autoencoders Using Importance Weighting

10/13/2020
by   Ghazi Felhi, et al.
0

Even though Variational Autoencoders (VAEs) are widely used for semi-supervised learning, the reason why they work remains unclear. In fact, the addition of the unsupervised objective is most often vaguely described as a regularization. The strength of this regularization is controlled by down-weighting the objective on the unlabeled part of the training set. Through an analysis of the objective of semi-supervised VAEs, we observe that they use the posterior of the learned generative model to guide the inference model in learning the partially observed latent variable. We show that given this observation, it is possible to gain finer control on the effect of the unsupervised objective on the training procedure. Using importance weighting, we derive two novel objectives that prioritize either one of the partially observed latent variable, or the unobserved latent variable. Experiments on the IMDB english sentiment analysis dataset and on the AG News topic classification dataset show the improvements brought by our prioritization mechanism and exhibit a behavior that is inline with our description of the inner working of Semi-Supervised VAEs.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/25/2022

Advancing Semi-Supervised Task Oriented Dialog Systems by JSA Learning of Discrete Latent Variable Models

Developing semi-supervised task-oriented dialog (TOD) systems by leverag...
research
09/27/2021

Challenging the Semi-Supervised VAE Framework for Text Classification

Semi-Supervised Variational Autoencoders (SSVAEs) are widely used models...
research
06/10/2018

A Structured Variational Autoencoder for Contextual Morphological Inflection

Statistical morphological inflectors are typically trained on fully supe...
research
06/07/2019

Importance Weighted Adversarial Variational Autoencoders for Spike Inference from Calcium Imaging Data

The Importance Weighted Auto Encoder (IWAE) objective has been shown to ...
research
07/23/2017

Prediction-Constrained Training for Semi-Supervised Mixture and Topic Models

Supervisory signals have the potential to make low-dimensional data repr...
research
01/16/2012

Spike-and-Slab Sparse Coding for Unsupervised Feature Discovery

We consider the problem of using a factor model we call spike-and-slab ...
research
10/03/2019

Semi-Supervised Generative Modeling for Controllable Speech Synthesis

We present a novel generative model that combines state-of-the-art neura...

Please sign up or login with your details

Forgot password? Click here to reset