Generative Adversarial Networks for Annotated Data Augmentation in Data Sparse NLU

12/09/2020
by   Olga Golovneva, et al.
0

Data sparsity is one of the key challenges associated with model development in Natural Language Understanding (NLU) for conversational agents. The challenge is made more complex by the demand for high quality annotated utterances commonly required for supervised learning, usually resulting in weeks of manual labor and high cost. In this paper, we present our results on boosting NLU model performance through training data augmentation using a sequential generative adversarial network (GAN). We explore data generation in the context of two tasks, the bootstrapping of a new language and the handling of low resource features. For both tasks we explore three sequential GAN architectures, one with a token-level reward function, another with our own implementation of a token-level Monte Carlo rollout reward, and a third with sentence-level reward. We evaluate the performance of these feedback models across several sampling methodologies and compare our results to upsampling the original data to the same scale. We further improve the GAN model performance through the transfer learning of the pretrained embeddings. Our experiments reveal synthetic data generated using the sequential generative adversarial network provides significant performance boosts across multiple metrics and can be a major benefit to the NLU tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/18/2019

Data augmentation for low resource sentiment analysis using generative adversarial networks

Sentiment analysis is a task that may suffer from a lack of data in cert...
research
04/10/2022

Generative Adversarial Networks for Image Augmentation in Agriculture: A Systematic Review

In agricultural image analysis, optimal model performance is keenly purs...
research
06/09/2023

Attention-stacked Generative Adversarial Network (AS-GAN)-empowered Sensor Data Augmentation for Online Monitoring of Manufacturing System

Machine learning (ML) has been extensively adopted for the online sensin...
research
10/29/2017

A Bayesian Data Augmentation Approach for Learning Deep Models

Data augmentation is an essential part of the training process applied t...
research
07/20/2023

Adversarial Conversational Shaping for Intelligent Agents

The recent emergence of deep learning methods has enabled the research c...
research
08/20/2021

Mitigating Greenhouse Gas Emissions Through Generative Adversarial Networks Based Wildfire Prediction

Over the past decade, the number of wildfire has increased significantly...

Please sign up or login with your details

Forgot password? Click here to reset