Skip-gram word embeddings in hyperbolic space

08/30/2018
by   Matthias Leimeister, et al.
0

Embeddings of tree-like graphs in hyperbolic space were recently shown to surpass their Euclidean counterparts in performance by a large margin. Inspired by these results, we present an algorithm for learning word embeddings in hyperbolic space from free text. An objective function based on the hyperbolic distance is derived and included in the skip-gram architecture from word2vec. The hyperbolic word embeddings are then evaluated on word similarity and analogy benchmarks. The results demonstrate the potential of hyperbolic word embeddings, particularly in low dimensions, though without clear superiority over their Euclidean counterparts. We further discuss problems in the formulation of the analogy task resulting from the curvature of hyperbolic space.

READ FULL TEXT
research
11/08/2022

Hyperbolic Centroid Calculations for Text Classification

A new development in NLP is the construction of hyperbolic word embeddin...
research
10/15/2018

Poincaré GloVe: Hyperbolic Word Embeddings

Words are not created equal. In fact, they form an aristocratic graph wi...
research
02/27/2020

Binarized PMI Matrix: Bridging Word Embeddings and Hyperbolic Spaces

We show analytically that removing sigmoid transformation in the SGNS ob...
research
12/11/2018

Hyperbolic Deep Learning for Chinese Natural Language Understanding

Recently hyperbolic geometry has proven to be effective in building embe...
research
05/26/2018

Stable Geodesic Update on Hyperbolic Space and its Application to Poincare Embeddings

A hyperbolic space has been shown to be more capable of modeling complex...
research
05/12/2023

Multi-Relational Hyperbolic Word Embeddings from Natural Language Definitions

Neural-based word embeddings using solely distributional information hav...
research
09/15/2021

Comparing Euclidean and Hyperbolic Embeddings on the WordNet Nouns Hypernymy Graph

Nickel and Kiela (2017) present a new method for embedding tree nodes in...

Please sign up or login with your details

Forgot password? Click here to reset