DeepAI AI Chat
Log In Sign Up

Evaluating Machine Translation Performance on Chinese Idioms with a Blacklist Method

11/21/2017
by   Yutong Shao, et al.
0

Idiom translation is a challenging problem in machine translation because the meaning of idioms is non-compositional, and a literal translation is likely to be wrong. In this paper, we assess the quality of idiom translation of a modern neural MT system. We introduce a new evaluation method based on an idiom-specific blacklist of literal translations, based on the insight that the occurrence of any blacklisted words in the translation output indicates a likely translation error. We introduce a dataset, CIBB (Chinese Idioms Blacklists Bank), and perform an evaluation of a state-of-the-art Chinese-English neural MT system. Our evaluation confirms that our blacklist method is effective at identifying literal translation errors, and that a sizable number of idioms in our test set are mistranslated (36.5

READ FULL TEXT

page 1

page 2

page 3

page 4

05/04/2018

Upping the Ante: Towards a Better Benchmark for Chinese-to-English Machine Translation

There are many machine translation (MT) papers that propose novel approa...
12/20/2022

Tackling Ambiguity with Images: Improved Multimodal Machine Translation and Contrastive Evaluation

One of the major challenges of machine translation (MT) is ambiguity, wh...
10/24/2021

Understanding the Impact of UGC Specificities on Translation Quality

This work takes a critical look at the evaluation of user-generated cont...
10/18/2015

Neural Reranking Improves Subjective Quality of Machine Translation: NAIST at WAT2015

This year, the Nara Institute of Science and Technology (NAIST)'s submis...
09/04/2019

SAO WMT19 Test Suite: Machine Translation of Audit Reports

This paper describes a machine translation test set of documents from th...
09/02/2018

MTNT: A Testbed for Machine Translation of Noisy Text

Noisy or non-standard input text can cause disastrous mistranslations in...
02/22/2022

An Overview on Machine Translation Evaluation

Since the 1950s, machine translation (MT) has become one of the importan...