Context-Aware Crosslingual Mapping

03/08/2019
by   Hanan Aldarmaki, et al.
0

Cross-lingual word vectors are typically obtained by fitting an orthogonal matrix that maps the entries of a bilingual dictionary from a source to a target vector space. Word vectors, however, are most commonly used for sentence or document-level representations that are calculated as the weighted average of word embeddings. In this paper, we propose an alternative to word-level mapping that better reflects sentence-level cross-lingual similarity. We incorporate context in the transformation matrix by directly mapping the averaged embeddings of aligned sentences in a parallel corpus. We also implement cross-lingual mapping of deep contextualized word embeddings using parallel sentences with word alignments. In our experiments, both approaches resulted in cross-lingual sentence embeddings that outperformed context-independent word mapping in sentence translation retrieval. Furthermore, the sentence-level transformation could be used for word-level mapping without loss in word translation quality.

READ FULL TEXT

page 1

page 2

page 3

page 4

03/08/2019

Context-Aware Cross-Lingual Mapping

Cross-lingual word vectors are typically obtained by fitting an orthogon...
12/28/2019

Robust Cross-lingual Embeddings from Parallel Sentences

Recent advances in cross-lingual word embeddings have primarily relied o...
04/07/2020

Locality Preserving Loss to Align Vector Spaces

We present a locality preserving loss (LPL)that improves the alignment b...
02/13/2017

Offline bilingual word vectors, orthogonal transformations and the inverted softmax

Usually bilingual word vectors are trained "online". Mikolov et al. show...
01/29/2020

ABSent: Cross-Lingual Sentence Representation Mapping with Bidirectional GANs

A number of cross-lingual transfer learning approaches based on neural n...
07/19/2019

Exploring sentence informativeness

This study is a preliminary exploration of the concept of informativenes...
08/18/2016

A Strong Baseline for Learning Cross-Lingual Word Embeddings from Sentence Alignments

While cross-lingual word embeddings have been studied extensively in rec...