Oracle-free Detection of Translation Issue for Neural Machine Translation

07/06/2018
by   Wujie Zheng, et al.
0

Neural Machine Translation (NMT) has been widely adopted over recent years due to its advantages on various translation tasks. However, NMT systems can be error-prone due to the intractability of natural languages and the design of neural networks, bringing issues to their translations. These issues could potentially lead to information loss, wrong semantics, and low readability in translations, compromising the usefulness of NMT and leading to potential non-trivial consequences. Although there are existing approaches, such as using the BLEU score, on quality assessment and issue detection for NMT, such approaches face two serious limitations. First, such solutions require oracle translations, i.e., reference translations, which are often unavailable, e.g., in production environments. Second, such approaches cannot pinpoint the issue types and locations within translations. To address such limitations, we propose a new approach aiming to precisely detect issues in translations without requiring oracle translations. Our approach focuses on two most prominent issues in NMT translations by including two detection algorithms. Our experimental results show that our new approach could achieve high effectiveness on real-world datasets. Our successful experience on deploying the proposed algorithms in both the development and production environments of WeChat, a messenger app with over one billion of monthly active users, helps eliminate numerous defects of our NMT model, monitor the effectiveness on real-world translation tasks, and collect in-house test cases, producing high industry impact.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/06/2018

Testing Untestable Neural Machine Translation: An Industrial Case

Neural Machine Translation (NMT) has been widely adopted recently due to...
research
10/17/2016

Neural Machine Translation Advised by Statistical Machine Translation

Neural Machine Translation (NMT) is a new approach to machine translatio...
research
09/19/2017

Dynamic Oracle for Neural Machine Translation in Decoding Phase

The past several years have witnessed the rapid progress of end-to-end N...
research
12/16/2021

Amortized Noisy Channel Neural Machine Translation

Noisy channel models have been especially effective in neural machine tr...
research
12/19/2022

Optimal Transport for Unsupervised Hallucination Detection in Neural Machine Translation

Neural machine translation (NMT) has become the de-facto standard in rea...
research
12/02/2019

Merging External Bilingual Pairs into Neural Machine Translation

As neural machine translation (NMT) is not easily amenable to explicit c...
research
03/15/2022

Can Synthetic Translations Improve Bitext Quality?

Synthetic translations have been used for a wide range of NLP tasks prim...

Please sign up or login with your details

Forgot password? Click here to reset