AI Chat AI Image Generator AI Video Text to Speech

Linking GloVe with word2vec

11/20/2014

∙

by Tianze Shi, et al.

∙

∙

The Global Vectors for word representation (GloVe), introduced by Jeffrey Pennington et al. is reported to be an efficient and effective method for learning vector representations of words. State-of-the-art performance is also provided by skip-gram with negative-sampling (SGNS) implemented in the word2vec tool. In this note, we explain the similarities between the training objectives of the two models, and show that the objective of SGNS is similar to the objective of a specialized form of GloVe, though their cost functions are defined differently.

Tianze Shi
16 publications
Zhiyuan Liu
209 publications

page 1

page 2

page 3

page 4

research

∙ 05/29/2020

InfiniteWalk: Deep Network Embeddings as Laplacian Embeddings with a Nonlinearity

The skip-gram model for learning word embeddings (Mikolov et al. 2013) h...

0 Sudhanshu Chanpuriya, et al. ∙

research

∙ 05/27/2017

word2vec Skip-Gram with Negative Sampling is a Weighted Logistic PCA

We show that the skip-gram formulation of word2vec trained with negative...

0 Andrew J. Landgraf, et al. ∙

research

∙ 02/15/2014

word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method

The word2vec software of Tomas Mikolov and colleagues (https://code.goog...

0 Yoav Goldberg, et al. ∙

research

∙ 04/01/2018

Revisiting Skip-Gram Negative Sampling Model with Regularization

We revisit skip-gram negative sampling (SGNS), a popular neural-network ...

0 Cun Mu, et al. ∙

research

∙ 06/06/2019

Second-order Co-occurrence Sensitivity of Skip-Gram with Negative Sampling

We simulate first- and second-order context overlap and show that Skip-G...

0 Dominik Schlechtweg, et al. ∙

research

∙ 10/16/2013

Distributed Representations of Words and Phrases and their Compositionality

The recently introduced continuous Skip-gram model is an efficient metho...

0 Tomas Mikolov, et al. ∙