DeepAI AI Chat
Log In Sign Up

On Compositional Generalization of Neural Machine Translation

by   Yafu Li, et al.

Modern neural machine translation (NMT) models have achieved competitive performance in standard benchmarks such as WMT. However, there still exist significant issues such as robustness, domain generalization, etc. In this paper, we study NMT models from the perspective of compositional generalization by building a benchmark dataset, CoGnition, consisting of 216k clean and consistent sentence pairs. We quantitatively analyze effects of various factors using compound translation error rate, then demonstrate that the NMT model fails badly on compositional generalization, although it performs remarkably well under traditional metrics.


Benchmarking Azerbaijani Neural Machine Translation

Little research has been done on Neural Machine Translation (NMT) for Az...

Categorizing Semantic Representations for Neural Machine Translation

Modern neural machine translation (NMT) models have achieved competitive...

Revisiting Iterative Back-Translation from the Perspective of Compositional Generalization

Human intelligence exhibits compositional generalization (i.e., the capa...

Detecting and Understanding Generalization Barriers for Neural Machine Translation

Generalization to unseen instances is our eternal pursuit for all data-d...

The Unreasonable Volatility of Neural Machine Translation Models

Recent works have shown that Neural Machine Translation (NMT) models ach...

Evaluating Explanation Methods for Neural Machine Translation

Recently many efforts have been devoted to interpreting the black-box NM...

PheMT: A Phenomenon-wise Dataset for Machine Translation Robustness on User-Generated Contents

Neural Machine Translation (NMT) has shown drastic improvement in its qu...

Code Repositories