DeepAI AI Chat
Log In Sign Up

Translation Error Detection as Rationale Extraction

08/27/2021
by   Marina Fomicheva, et al.
The University of Sheffield
0

Recent Quality Estimation (QE) models based on multilingual pre-trained representations have achieved very competitive results when predicting the overall quality of translated sentences. Predicting translation errors, i.e. detecting specifically which words are incorrect, is a more challenging task, especially with limited amounts of training data. We hypothesize that, not unlike humans, successful QE models rely on translation errors to predict overall sentence quality. By exploring a set of feature attribution methods that assign relevance scores to the inputs to explain model predictions, we study the behaviour of state-of-the-art sentence-level QE models and show that explanations (i.e. rationales) extracted from these models can indeed be used to detect translation errors. We therefore (i) introduce a novel semi-supervised method for word-level QE and (ii) propose to use the QE task as a new benchmark for evaluating the plausibility of feature attribution, i.e. how interpretable model explanations are to humans.

READ FULL TEXT
11/17/2022

Reducing Hallucinations in Neural Machine Translation with Feature Attribution

Neural conditional language generation models achieve the state-of-the-a...
06/19/2020

How does this interaction affect me? Interpretable attribution for feature interactions

Machine learning transparency calls for interpretable explanations of ho...
11/08/2018

Looking Deeper into Deep Learning Model: Attribution-based Explanations of TextCNN

Layer-wise Relevance Propagation (LRP) and saliency maps have been recen...
04/11/2022

A Multilingual Perspective Towards the Evaluation of Attribution Methods in Natural Language Inference

Most evaluations of attribution methods focus on the English language. I...
05/25/2021

IntelliCAT: Intelligent Machine Translation Post-Editing with Quality Estimation and Translation Suggestion

We present IntelliCAT, an interactive translation interface with neural ...
02/22/2019

OpenKiwi: An Open Source Framework for Quality Estimation

We introduce OpenKiwi, a Pytorch-based open source framework for transla...
09/13/2022

CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Task

We present the joint contribution of IST and Unbabel to the WMT 2022 Sha...