SDA: Improving Text Generation with Self Data Augmentation

01/02/2021
by   Ping Yu, et al.
0

Data augmentation has been widely used to improve deep neural networks in many research fields, such as computer vision. However, less work has been done in the context of text, partially due to its discrete nature and the complexity of natural languages. In this paper, we propose to improve the standard maximum likelihood estimation (MLE) paradigm by incorporating a self-imitation-learning phase for automatic data augmentation. Unlike most existing sentence-level augmentation strategies, which are only applied to specific models, our method is more general and could be easily adapted to any MLE-based training procedure. In addition, our framework allows task-specific evaluation metrics to be designed to flexibly control the generated sentences, for example, in terms of controlling vocabulary usage and avoiding nontrivial repetitions. Extensive experimental results demonstrate the superiority of our method on two synthetic and several standard real datasets, significantly improving related baselines.

READ FULL TEXT
05/28/2021

Data Augmentation for Text Generation Without Any Augmented Data

Data augmentation is an effective way to improve the performance of many...
07/12/2020

Improving Maximum Likelihood Training for Text Generation with Density Ratio Estimation

Auto-regressive sequence generative models trained by Maximum Likelihood...
07/14/2020

Deep Transformer based Data Augmentation with Subword Units for Morphologically Rich Online ASR

Recently Deep Transformer models have proven to be particularly powerful...
03/25/2021

Adversarial Imitation Learning with Trajectorial Augmentation and Correction

Deep Imitation Learning requires a large number of expert demonstrations...
05/25/2019

Soft Contextual Data Augmentation for Neural Machine Translation

While data augmentation is an important trick to boost the accuracy of d...
11/08/2021

Off-policy Imitation Learning from Visual Inputs

Recently, various successful applications utilizing expert states in imi...
04/26/2018

An Integrative Introduction to Human Augmentation Science

Human Augmentation (HA) spans several technical fields and methodologica...