Fine-Grained Attention Mechanism for Neural Machine Translation

03/30/2018
by   Heeyoul Choi, et al.
0

Neural machine translation (NMT) has been a new paradigm in machine translation, and the attention mechanism has become the dominant approach with the state-of-the-art records in many language pairs. While there are variants of the attention mechanism, all of them use only temporal attention where one scalar value is assigned to one context vector corresponding to a source word. In this paper, we propose a fine-grained (or 2D) attention mechanism where each dimension of a context vector will receive a separate attention score. In experiments with the task of En-De and En-Fi translation, the fine-grained attention method improves the translation quality in terms of BLEU score. In addition, our alignment analysis reveals how the fine-grained attention mechanism exploits the internal structure of context vectors.

READ FULL TEXT

page 7

page 8

research
01/19/2016

Modeling Coverage for Neural Machine Translation

Attention mechanism has enhanced state-of-the-art Neural Machine Transla...
research
11/10/2019

Modelling Bahdanau Attention using Election methods aided by Q-Learning

Neural Machine Translation has lately gained a lot of "attention" with t...
research
02/17/2023

Towards Fine-Grained Information: Identifying the Type and Location of Translation Errors

Fine-grained information on translation errors is helpful for the transl...
research
04/05/2020

Detecting and Understanding Generalization Barriers for Neural Machine Translation

Generalization to unseen instances is our eternal pursuit for all data-d...
research
09/13/2016

Multimodal Attention for Neural Machine Translation

The attention mechanism is an important part of the neural machine trans...
research
04/28/2022

Attention Mechanism with Energy-Friendly Operations

Attention mechanism has become the dominant module in natural language p...
research
03/13/2020

Sentence Level Human Translation Quality Estimation with Attention-based Neural Networks

This paper explores the use of Deep Learning methods for automatic estim...

Please sign up or login with your details

Forgot password? Click here to reset