Log In Sign Up

Out-of-Vocabulary Entities in Link Prediction

by   Caglar Demir, et al.

Knowledge graph embedding techniques are key to making knowledge graphs amenable to the plethora of machine learning approaches based on vector representations. Link prediction is often used as a proxy to evaluate the quality of these embeddings. Given that the creation of benchmarks for link prediction is a time-consuming endeavor, most work on the subject matter uses only a few benchmarks. As benchmarks are crucial for the fair comparison of algorithms, ensuring their quality is tantamount to providing a solid ground for developing better solutions to link prediction and ipso facto embedding knowledge graphs. First studies of benchmarks pointed to limitations pertaining to information leaking from the development to the test fragments of some benchmark datasets. We spotted a further common limitation of three of the benchmarks commonly used for evaluating link prediction approaches: out-of-vocabulary entities in the test and validation sets. We provide an implementation of an approach for spotting and removing such entities and provide corrected versions of the datasets WN18RR, FB15K-237, and YAGO3-10. Our experiments on the corrected versions of WN18RR, FB15K-237, and YAGO3-10 suggest that the measured performance of state-of-the-art approaches is altered significantly with p-values <1 state-of-the-art approaches gain on average absolute 3.29 ± 0.24% in all metrics on WN18RR. This means that some of the conclusions achieved in previous works might need to be revisited. We provide an open-source implementation of our experiments and corrected datasets at at


page 1

page 2

page 3

page 4


Trust from the past: Bayesian Personalized Ranking based Link Prediction in Knowledge Graphs

Link prediction, or predicting the likelihood of a link in a knowledge g...

Toward perfect reads

We propose a new method to correct short reads using de Bruijn graphs, a...

Convolutional Hypercomplex Embeddings for Link Prediction

Knowledge graph embedding research has mainly focused on the two smalles...

Convolutional Complex Knowledge Graph Embeddings

In this paper, we study the problem of learning continuous vector repres...

Kronecker Decomposition for Knowledge Graph Embeddings

Knowledge graph embedding research has mainly focused on learning contin...

Global and local evaluation of link prediction tasks with neural embeddings

We focus our attention on the link prediction problem for knowledge grap...