Data Ordering Patterns for Neural Machine Translation: An Empirical Study

09/23/2019
by   Siddhant Garg, et al.
0

Recent works show that ordering of the training data affects the model performance for Neural Machine Translation. Several approaches involving dynamic data ordering and data sharding based on curriculum learning have been analysed for the their performance gains and faster convergence. In this work we propose to empirically study several ordering approaches for the training data based on different metrics and evaluate their impact on the model performance. Results from our study show that pre-fixing the ordering of the training data based on perplexity scores from a pre-trained model performs the best and outperforms the default approach of randomly shuffling the training data every epoch.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/25/2022

Data Selection Curriculum for Neural Machine Translation

Neural Machine Translation (NMT) models are typically trained on heterog...
research
04/13/2020

Reinforced Curriculum Learning on Pre-trained Neural Machine Translation Models

The competitive performance of neural machine translation (NMT) critical...
research
06/16/2021

TSO: Curriculum Generation using continuous optimization

The training of deep learning models poses vast challenges of including ...
research
09/09/2019

Does Order Matter? An Empirical Study on Generating Multiple Keyphrases as a Sequence

Recently, concatenating multiple keyphrases as a target sequence has bee...
research
11/04/2019

On Compositionality in Neural Machine Translation

We investigate two specific manifestations of compositionality in Neural...
research
10/07/2020

Dual Reconstruction: a Unifying Objective for Semi-Supervised Neural Machine Translation

While Iterative Back-Translation and Dual Learning effectively incorpora...
research
02/28/2022

LCP-dropout: Compression-based Multiple Subword Segmentation for Neural Machine Translation

In this study, we propose a simple and effective preprocessing method fo...

Please sign up or login with your details

Forgot password? Click here to reset