DeepAI AI Chat
Log In Sign Up

Practical Perspectives on Quality Estimation for Machine Translation

by   Junpei Zhou, et al.
Carnegie Mellon University

Sentence level quality estimation (QE) for machine translation (MT) attempts to predict the translation edit rate (TER) cost of post-editing work required to correct MT output. We describe our view on sentence-level QE as dictated by several practical setups encountered in the industry. We find consumers of MT output—whether human or algorithmic ones—to be primarily interested in a binary quality metric: is the translated sentence adequate as-is or does it need post-editing? Motivated by this we propose a quality classification (QC) view on sentence-level QE whereby we focus on maximizing recall at precision above a given threshold. We demonstrate that, while classical QE regression models fare poorly on this task, they can be re-purposed by replacing the output regression layer with a binary classification one, achieving 50-60% recall at 90% precision. For a high-quality MT system producing 75-80% correct translations, this promises a significant reduction in post-editing work indeed.


page 1

page 2

page 3

page 4


MLQE-PE: A Multilingual Quality Estimation and Post-Editing Dataset

We present MLQE-PE, a new dataset for Machine Translation (MT) Quality E...

Sentence-level quality estimation by predicting HTER as a multi-component metric

This submission investigates alternative machine learning models for pre...

Fluency Over Adequacy: A Pilot Study in Measuring User Trust in Imperfect MT

Although measuring intrinsic quality has been a key factor in the advanc...

Ensemble Fine-tuned mBERT for Translation Quality Estimation

Quality Estimation (QE) is an important component of the machine transla...

Automatic Post-Editing for Machine Translation

Automatic Post-Editing (APE) aims to correct systematic errors in a mach...

AlphaMWE: Construction of Multilingual Parallel Corpora with MWE Annotations

In this work, we present the construction of multilingual parallel corpo...