Out-of-Vocabulary Entities in Link Prediction

05/26/2021
by   Caglar Demir, et al.
0

Knowledge graph embedding techniques are key to making knowledge graphs amenable to the plethora of machine learning approaches based on vector representations. Link prediction is often used as a proxy to evaluate the quality of these embeddings. Given that the creation of benchmarks for link prediction is a time-consuming endeavor, most work on the subject matter uses only a few benchmarks. As benchmarks are crucial for the fair comparison of algorithms, ensuring their quality is tantamount to providing a solid ground for developing better solutions to link prediction and ipso facto embedding knowledge graphs. First studies of benchmarks pointed to limitations pertaining to information leaking from the development to the test fragments of some benchmark datasets. We spotted a further common limitation of three of the benchmarks commonly used for evaluating link prediction approaches: out-of-vocabulary entities in the test and validation sets. We provide an implementation of an approach for spotting and removing such entities and provide corrected versions of the datasets WN18RR, FB15K-237, and YAGO3-10. Our experiments on the corrected versions of WN18RR, FB15K-237, and YAGO3-10 suggest that the measured performance of state-of-the-art approaches is altered significantly with p-values <1 state-of-the-art approaches gain on average absolute 3.29 ± 0.24% in all metrics on WN18RR. This means that some of the conclusions achieved in previous works might need to be revisited. We provide an open-source implementation of our experiments and corrected datasets at at https://github.com/dice-group/OOV-In-Link-Prediction.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/14/2016

Trust from the past: Bayesian Personalized Ranking based Link Prediction in Knowledge Graphs

Link prediction, or predicting the likelihood of a link in a knowledge g...
research
08/07/2020

Convolutional Complex Knowledge Graph Embeddings

In this paper, we study the problem of learning continuous vector repres...
research
06/29/2021

Convolutional Hypercomplex Embeddings for Link Prediction

Knowledge graph embedding research has mainly focused on the two smalles...
research
11/09/2017

Toward perfect reads

We propose a new method to correct short reads using de Bruijn graphs, a...
research
05/13/2022

Kronecker Decomposition for Knowledge Graph Embeddings

Knowledge graph embedding research has mainly focused on learning contin...
research
02/03/2020

Knowledge Graph Embedding for Link Prediction: A Comparative Analysis

Knowledge Graphs (KGs) have found many applications in industry and acad...

Please sign up or login with your details

Forgot password? Click here to reset