Graph Convolutional Encoders for Syntax-aware Neural Machine Translation

by   Joost Bastings, et al.
University of Amsterdam

We present a simple and effective approach to incorporating syntactic structure into neural attention-based encoder-decoder models for machine translation. We rely on graph-convolutional networks (GCNs), a recent class of neural networks developed for modeling graph-structured data. Our GCNs use predicted syntactic dependency trees of source sentences to produce representations of words (i.e. hidden states of the encoder) that are sensitive to their syntactic neighborhoods. GCNs take word representations as input and produce word representations as output, so they can easily be incorporated as layers into standard encoders (e.g., on top of bidirectional RNNs or convolutional neural networks). We evaluate their effectiveness with English-German and English-Czech translation experiments for different types of encoders and observe substantial improvements over their syntax-agnostic versions in all the considered setups.


page 1

page 2

page 3

page 4


Exploiting Semantics in Neural Machine Translation with Graph Convolutional Networks

Semantic representations have long been argued as potentially useful for...

Neural Machine Translation with Dynamic Graph Convolutional Decoder

Existing wisdom demonstrates the significance of syntactic knowledge for...

Multi-Source Syntactic Neural Machine Translation

We introduce a novel multi-source technique for incorporating source syn...

Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling

Semantic role labeling (SRL) is the task of identifying the predicate-ar...

Recurrent Graph Syntax Encoder for Neural Machine Translation

Syntax-incorporated machine translation models have been proven successf...

Transition based Graph Decoder for Neural Machine Translation

While a number of works showed gains from incorporating source-side symb...

Compositional Neural Machine Translation by Removing the Lexicon from Syntax

The meaning of a natural language utterance is largely determined from i...

Please sign up or login with your details

Forgot password? Click here to reset