Word, graph and manifold embedding from Markov processes

09/18/2015
by   Tatsunori B. Hashimoto, et al.
0

Continuous vector representations of words and objects appear to carry surprisingly rich semantic content. In this paper, we advance both the conceptual and theoretical understanding of word embeddings in three ways. First, we ground embeddings in semantic spaces studied in cognitive-psychometric literature and introduce new evaluation tasks. Second, in contrast to prior work, we take metric recovery as the key object of study, unify existing algorithms as consistent metric recovery methods based on co-occurrence counts from simple Markov random walks, and propose a new recovery algorithm. Third, we generalize metric recovery to graphs and manifolds, relating co-occurence counts on random walks in graphs and random processes on manifolds to the underlying metric to be recovered, thereby reconciling manifold estimation and embedding algorithms. We compare embedding algorithms across a range of tasks, from nonlinear dimensionality reduction to three semantic language tasks, including analogies, sequence completion, and classification.

READ FULL TEXT
research
07/18/2016

Language classification from bilingual word embedding graphs

We study the role of the second language in bilingual word embeddings in...
research
05/16/2020

RPD: A Distance Function Between Word Embeddings

It is well-understood that different algorithms, training processes, and...
research
04/05/2022

Walk this Way! Entity Walks and Property Walks for RDF2vec

RDF2vec is a knowledge graph embedding mechanism which first extracts se...
research
12/16/2014

Rehabilitation of Count-based Models for Word Vector Representations

Recent works on word representations mostly rely on predictive models. D...
research
02/07/2018

Learning Role-based Graph Embeddings

Random walks are at the heart of many existing network embedding methods...
research
02/02/2022

Efficient Random Walks on Riemannian Manifolds

According to a version of Donsker's theorem, geodesic random walks on Ri...
research
07/26/2023

The flow of ideas in word embeddings

The flow of ideas has been extensively studied by physicists, psychologi...

Please sign up or login with your details

Forgot password? Click here to reset