Discrete Latent Variable Representations for Low-Resource Text Classification

06/11/2020
by   Shuning Jin, et al.
0

While much work on deep latent variable models of text uses continuous latent variables, discrete latent variables are interesting because they are more interpretable and typically more space efficient. We consider several approaches to learning discrete latent variable models for text in the case where exact marginalization over these variables is intractable. We compare the performance of the learned representations as features for low-resource document and sentence classification. Our best models outperform the previous best reported results with continuous representations in these low-resource settings, while learning significantly more compressed representations. Interestingly, we find that an amortized variant of Hard EM performs particularly well in the lowest-resource regimes.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/12/2020

Truncated Inference for Latent Variable Optimization Problems: Application to Robust Estimation and Learning

Optimization problems with an auxiliary latent variable structure in add...
research
11/14/2018

Extractive Summary as Discrete Latent Variables

In this paper, we compare various methods to compress a text using a neu...
research
10/29/2022

Learning Dependencies of Discrete Speech Representations with Neural Hidden Markov Models

While discrete latent variable models have had great success in self-sup...
research
10/01/2019

Latent-Variable Generative Models for Data-Efficient Text Classification

Generative classifiers offer potential advantages over their discriminat...
research
11/07/2022

Learning Semantic Textual Similarity via Topic-informed Discrete Latent Variables

Recently, discrete latent variable models have received a surge of inter...
research
08/28/2023

NAS-X: Neural Adaptive Smoothing via Twisting

We present Neural Adaptive Smoothing via Twisting (NAS-X), a method for ...

Please sign up or login with your details

Forgot password? Click here to reset