Binarized PMI Matrix: Bridging Word Embeddings and Hyperbolic Spaces

02/27/2020
by   Zhenisbek Assylbekov, et al.
0

We show analytically that removing sigmoid transformation in the SGNS objective does not harm the quality of word vectors significantly and at the same time is related to factorizing a binarized PMI matrix which, in turn, can be treated as an adjacency matrix of a certain graph. Empirically, such graph is a complex network, i.e. it has strong clustering and scale-free degree distribution, and is tightly connected with hyperbolic spaces. In short, we show the connection between static word embeddings and hyperbolic spaces through the binarized PMI matrix using analytical and empirical methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/30/2018

Skip-gram word embeddings in hyperbolic space

Embeddings of tree-like graphs in hyperbolic space were recently shown t...
research
11/08/2022

Hyperbolic Centroid Calculations for Text Classification

A new development in NLP is the construction of hyperbolic word embeddin...
research
04/26/2022

From Hyperbolic Geometry Back to Word Embeddings

We choose random points in the hyperbolic disc and claim that these poin...
research
10/15/2018

Poincaré GloVe: Hyperbolic Word Embeddings

Words are not created equal. In fact, they form an aristocratic graph wi...
research
03/09/2017

What can you do with a rock? Affordance extraction via word embeddings

Autonomous agents must often detect affordances: the set of behaviors en...
research
04/18/2019

Analytical Methods for Interpretable Ultradense Word Embeddings

Word embeddings are useful for a wide variety of tasks, but they lack in...
research
02/26/2019

Context Vectors are Reflections of Word Vectors in Half the Dimensions

This paper takes a step towards theoretical analysis of the relationship...

Please sign up or login with your details

Forgot password? Click here to reset