A Convolutional Encoder Model for Neural Machine Translation

11/07/2016
by   Jonas Gehring, et al.
0

The prevalent approach to neural machine translation relies on bi-directional LSTMs to encode the source sentence. In this paper we present a faster and simpler architecture based on a succession of convolutional layers. This allows to encode the entire source sentence simultaneously compared to recurrent networks for which computation is constrained by temporal dependencies. On WMT'16 English-Romanian translation we achieve competitive accuracy to the state-of-the-art and we outperform several recently published results on the WMT'15 English-German task. Our models obtain almost the same accuracy as a very deep LSTM setup on WMT'14 English-French translation. Our convolutional encoder speeds up CPU decoding by more than two times at the same or higher accuracy as a strong bi-directional LSTM baseline.

READ FULL TEXT

page 12

page 13

12/26/2018

Learning to Refine Source Representations for Neural Machine Translation

Neural machine translation (NMT) models generally adopt an encoder-decod...
12/05/2014

On Using Very Large Target Vocabulary for Neural Machine Translation

Neural machine translation, a recently proposed approach to machine tran...
10/01/2016

Vocabulary Selection Strategies for Neural Machine Translation

Classical translation models constrain the space of possible outputs by ...
01/18/2019

Modeling Latent Sentence Structure in Neural Machine Translation

Recently it was shown that linguistic structure predicted by a supervise...
05/04/2017

Sharp Models on Dull Hardware: Fast and Accurate Neural Machine Translation Decoding on the CPU

Attentional sequence-to-sequence models have become the new standard for...
10/31/2016

Neural Machine Translation in Linear Time

We present a novel neural network for processing sequences. The ByteNet ...
05/07/2018

Sentence-State LSTM for Text Representation

Bi-directional LSTMs are a powerful tool for text representation. On the...

Code Repositories

fairseq

Facebook AI Research Sequence-to-Sequence Toolkit


view repo