Binarized PMI Matrix: Bridging Word Embeddings and Hyperbolic Spaces

02/27/2020
by   Zhenisbek Assylbekov, et al.
0

We show analytically that removing sigmoid transformation in the SGNS objective does not harm the quality of word vectors significantly and at the same time is related to factorizing a binarized PMI matrix which, in turn, can be treated as an adjacency matrix of a certain graph. Empirically, such graph is a complex network, i.e. it has strong clustering and scale-free degree distribution, and is tightly connected with hyperbolic spaces. In short, we show the connection between static word embeddings and hyperbolic spaces through the binarized PMI matrix using analytical and empirical methods.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset