Convolutional Sequence to Sequence Learning

05/08/2017
by   Jonas Gehring, et al.
0

The prevalent approach to sequence to sequence learning maps an input sequence to a variable length output sequence via recurrent neural networks. We introduce an architecture based entirely on convolutional neural networks. Compared to recurrent models, computations over all elements can be fully parallelized during training and optimization is easier since the number of non-linearities is fixed and independent of the input length. Our use of gated linear units eases gradient propagation and we equip each decoder layer with a separate attention module. We outperform the accuracy of the deep LSTM setup of Wu et al. (2016) on both WMT'14 English-German and WMT'14 English-French translation at an order of magnitude faster speed, both on GPU and CPU.

READ FULL TEXT
01/11/2021

Context- and Sequence-Aware Convolutional Recurrent Encoder for Neural Machine Translation

Neural Machine Translation model is a sequence-to-sequence converter bas...
07/02/2018

Dynamic Prediction Length for Time Series with Sequence to Sequence Networks

Recurrent neural networks and sequence to sequence models require a pred...
09/15/2020

Recurrent autoencoder with sequence-aware encoding

Recurrent Neural Networks (RNN) received a vast amount of attention last...
06/09/2015

Pointer Networks

We introduce a new neural architecture to learn the conditional probabil...
11/14/2017

Classical Structured Prediction Losses for Sequence to Sequence Learning

There has been much recent work on training neural attention models at t...
02/11/2021

Variational Bayesian Sequence-to-Sequence Networks for Memory-Efficient Sign Language Translation

Memory-efficient continuous Sign Language Translation is a significant c...
12/27/2017

CNN Is All You Need

The Convolution Neural Network (CNN) has demonstrated the unique advanta...

Code Repositories

fairseq

Facebook AI Research Sequence-to-Sequence Toolkit


view repo

neurowriter

Framework to imitate writing styles using deep learning


view repo

fairseq-translator

A quick Tensorflow implementation of Facebook FairSeq[1] for character-level neural machine translation (EN -> JP).


view repo