RelWalk A Latent Variable Model Approach to Knowledge Graph Embedding

by   Danushka Bollegala, et al.

Embedding entities and relations of a knowledge graph in a low-dimensional space has shown impressive performance in predicting missing links between entities. Although progresses have been achieved, existing methods are heuristically motivated and theoretical understanding of such embeddings is comparatively underdeveloped. This paper extends the random walk model (Arora et al., 2016a) of word embeddings to Knowledge Graph Embeddings (KGEs) to derive a scoring function that evaluates the strength of a relation R between two entities h (head) and t (tail). Moreover, we show that marginal loss minimisation, a popular objective used in much prior work in KGE, follows naturally from the log-likelihood ratio maximisation under the probabilities estimated from the KGEs according to our theoretical relationship. We propose a learning objective motivated by the theoretical analysis to learn KGEs from a given knowledge graph. Using the derived objective, accurate KGEs are learnt from FB15K237 and WN18RR benchmark datasets, providing empirical evidence in support of the theory.


page 1

page 2

page 3

page 4


Composing Knowledge Graph Embeddings via Word Embeddings

Learning knowledge graph embedding from an existing knowledge graph is v...

What is Learned in Knowledge Graph Embeddings?

A knowledge graph (KG) is a data structure which represents entities and...

Group Representation Theory for Knowledge Graph Embedding

Knowledge graph embedding has recently become a popular way to model rel...

A Group-Theoretic Framework for Knowledge Graph Embedding

We demonstrated the existence of a group algebraic structure hidden in r...

AutoKGE: Searching Scoring Functions for Knowledge Graph Embedding

Knowledge graph embedding (KGE) aims to find low dimensional vector repr...

Comprehensive Analysis of Negative Sampling in Knowledge Graph Representation Learning

Negative sampling (NS) loss plays an important role in learning knowledg...

ProjE: Embedding Projection for Knowledge Graph Completion

With the large volume of new information created every day, determining ...