What is Learned in Knowledge Graph Embeddings?

by   Michael R. Douglas, et al.

A knowledge graph (KG) is a data structure which represents entities and relations as the vertices and edges of a directed graph with edge types. KGs are an important primitive in modern machine learning and artificial intelligence. Embedding-based models, such as the seminal TransE [Bordes et al., 2013] and the recent PairRE [Chao et al., 2020] are among the most popular and successful approaches for representing KGs and inferring missing edges (link completion). Their relative success is often credited in the literature to their ability to learn logical rules between the relations. In this work, we investigate whether learning rules between relations is indeed what drives the performance of embedding-based methods. We define motif learning and two alternative mechanisms, network learning (based only on the connectivity of the KG, ignoring the relation types), and unstructured statistical learning (ignoring the connectivity of the graph). Using experiments on synthetic KGs, we show that KG models can learn motifs and how this ability is degraded by non-motif (noise) edges. We propose tests to distinguish the contributions of the three mechanisms to performance, and apply them to popular KG benchmarks. We also discuss an issue with the standard performance testing protocol and suggest an improvement. To appear in the proceedings of Complex Networks 2021.


page 1

page 2

page 3

page 4


RelWalk A Latent Variable Model Approach to Knowledge Graph Embedding

Embedding entities and relations of a knowledge graph in a low-dimension...

Embedding Entities and Relations for Learning and Inference in Knowledge Bases

We consider learning representations of entities and relations in KBs us...

CoKE: Contextualized Knowledge Graph Embedding

Knowledge graph embedding, which projects symbolic entities and relation...

Knowledge Graph Completion with Text-aided Regularization

Knowledge Graph Completion is a task of expanding the knowledge graph/ba...

New Embedded Representations and Evaluation Protocols for Inferring Transitive Relations

Beyond word embeddings, continuous representations of knowledge graph (K...

On Understanding Knowledge Graph Representation

Many methods have been developed to represent knowledge graph data, whic...

Graph Learning for Inverse Landscape Genetics

The problem of inferring unknown graph edges from numerical data at a gr...