DeepAI AI Chat
Log In Sign Up

Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classification

01/26/2021
by   Yi Zhu, et al.
4

Semi-supervised learning through deep generative models and multi-lingual pretraining techniques have orchestrated tremendous success across different areas of NLP. Nonetheless, their development has happened in isolation, while the combination of both could potentially be effective for tackling task-specific labelled data shortage. To bridge this gap, we combine semi-supervised deep generative models and multi-lingual pretraining to form a pipeline for document classification task. Compared to strong supervised learning baselines, our semi-supervised classification framework is highly competitive and outperforms the state-of-the-art counterparts in low-resource settings across several languages.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 6

page 8

page 9

page 15

02/17/2016

Auxiliary Deep Generative Models

Deep generative models parameterized by neural networks have recently ac...
11/22/2016

Max-Margin Deep Generative Models for (Semi-)Supervised Learning

Deep generative models (DGMs) are effective on learning multilayered rep...
04/07/2022

A Joint Learning Approach for Semi-supervised Neural Topic Modeling

Topic models are some of the most popular ways to represent textual data...
06/01/2018

Learning Neural Random Fields with Inclusive Auxiliary Generators

In this paper we develop Neural Random Field learning with Inclusive-div...
12/12/2020

Learning Consistent Deep Generative Models from Sparse Data via Prediction Constraints

We develop a new framework for learning variational autoencoders and oth...
06/29/2017

Bayesian Semisupervised Learning with Deep Generative Models

Neural network based generative models with discriminative components ar...
06/05/2019

Variational Pretraining for Semi-supervised Text Classification

We introduce VAMPIRE, a lightweight pretraining framework for effective ...