DeepAI AI Chat
Log In Sign Up

Leveraging Monolingual Data for Crosslingual Compositional Word Representations

by   Hubert Soyer, et al.
National Institute of Informatics

In this work, we present a novel neural network based architecture for inducing compositional crosslingual word representations. Unlike previously proposed methods, our method fulfills the following three criteria; it constrains the word-level representations to be compositional, it is capable of leveraging both bilingual and monolingual data, and it is scalable to large vocabularies and large quantities of data. The key component of our approach is what we refer to as a monolingual inclusion criterion, that exploits the observation that phrases are more closely semantically related to their sub-phrases than to other randomly sampled phrases. We evaluate our method on a well-established crosslingual document classification task and achieve results that are either comparable, or greatly improve upon previous state-of-the-art methods. Concretely, our method reaches a level of 92.7 the English to German and German to English sub-tasks respectively. The former advances the state of the art by 0.9 absolute improvement upon the previous state of the art by 7.7 accuracy and an improvement of 33.0


page 1

page 2

page 3

page 4


Adaptive Joint Learning of Compositional and Non-Compositional Phrase Embeddings

We present a novel method for jointly learning compositional and non-com...

A Multivariate Model for Representing Semantic Non-compositionality

Semantically non-compositional phrases constitute an intriguing research...

Self-Training Sampling with Monolingual Data Uncertainty for Neural Machine Translation

Self-training has proven effective for improving NMT performance by augm...

A Sub-Character Architecture for Korean Language Processing

We introduce a novel sub-character architecture that exploits a unique c...

Word and Phrase Translation with word2vec

Word and phrase tables are key inputs to machine translations, but costl...

Keyphrase Generation with Correlation Constraints

In this paper, we study automatic keyphrase generation. Although convent...