Semi-supervised Sequential Generative Models

06/30/2020
by   Michael Teng, et al.
10

We introduce a novel objective for training deep generative time-series models with discrete latent variables for which supervision is only sparsely available. This instance of semi-supervised learning is challenging for existing methods, because the exponential number of possible discrete latent configurations results in high variance gradient estimators. We first overcome this problem by extending the standard semi-supervised generative modeling objective with reweighted wake-sleep. However, we find that this approach still suffers when the frequency of available labels varies between training sequences. Finally, we introduce a unified objective inspired by teacher-forcing and show that this approach is robust to variable length supervision. We call the resulting method caffeinated wake-sleep (CWS) to emphasize its additional dependence on real data. We demonstrate its effectiveness with experiments on MNIST, handwriting, and fruit fly trajectory data.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 4

page 8

02/17/2016

Auxiliary Deep Generative Models

Deep generative models parameterized by neural networks have recently ac...
01/24/2019

Semi-Unsupervised Learning with Deep Generative Models: Clustering and Classifying using Ultra-Sparse Labels

We introduce semi-unsupervised learning, an extreme case of semi-supervi...
06/23/2019

Variational Sequential Labelers for Semi-Supervised Learning

We introduce a family of multitask variational methods for semi-supervis...
01/26/2021

Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classification

Semi-supervised learning through deep generative models and multi-lingua...
10/03/2019

Semi-Supervised Generative Modeling for Controllable Speech Synthesis

We present a novel generative model that combines state-of-the-art neura...
09/23/2016

Language as a Latent Variable: Discrete Generative Models for Sentence Compression

In this work we explore deep generative models of text in which the late...
07/23/2017

Prediction-Constrained Training for Semi-Supervised Mixture and Topic Models

Supervisory signals have the potential to make low-dimensional data repr...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.