On the Convergent Properties of Word Embedding Methods

05/12/2016
by   Yingtao Tian, et al.
0

Do word embeddings converge to learn similar things over different initializations? How repeatable are experiments with word embeddings? Are all word embedding techniques equally reliable? In this paper we propose evaluating methods for learning word representations by their consistency across initializations. We propose a measure to quantify the similarity of the learned word representations under this setting (where they are subject to different random initializations). Our preliminary results illustrate that our metric not only measures a intrinsic property of word embedding methods but also correlates well with other evaluation metrics on downstream tasks. We believe our methods are is useful in characterizing robustness -- an important property to consider when developing new word embedding methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/28/2019

Evaluating Word Embedding Models: Methods and Experimental Results

Extensive evaluation on a large number of word embedding models for lang...
research
04/25/2018

Factors Influencing the Surprising Instability of Word Embeddings

Despite the recent popularity of word embedding methods, there is only a...
research
04/11/2018

Evaluating Word Embedding Hyper-Parameters for Similarity and Analogy Tasks

The versatility of word embeddings for various applications is attractin...
research
09/12/2019

Retrofitting Contextualized Word Embeddings with Paraphrases

Contextualized word embedding models, such as ELMo, generate meaningful ...
research
06/16/2022

TransDrift: Modeling Word-Embedding Drift using Transformer

In modern NLP applications, word embeddings are a crucial backbone that ...
research
10/04/2016

Are Word Embedding-based Features Useful for Sarcasm Detection?

This paper makes a simple increment to state-of-the-art in sarcasm detec...
research
02/15/2020

Supervised Phrase-boundary Embeddings

We propose a new word embedding model, called SPhrase, that incorporates...

Please sign up or login with your details

Forgot password? Click here to reset