MLMLM: Link Prediction with Mean Likelihood Masked Language Model

by   Louis Clouâtre, et al.

Knowledge Bases (KBs) are easy to query, verifiable, and interpretable. They however scale with man-hours and high-quality data. Masked Language Models (MLMs), such as BERT, scale with computing power as well as unstructured raw text data. The knowledge contained within those models is however not directly interpretable. We propose to perform link prediction with MLMs to address both the KBs scalability issues and the MLMs interpretability issues. To do that we introduce MLMLM, Mean Likelihood Masked Language Model, an approach comparing the mean likelihood of generating the different entities to perform link prediction in a tractable manner. We obtain State of the Art (SotA) results on the WN18RR dataset and the best non-entity-embedding based results on the FB15k-237 dataset. We also obtain convincing results on link prediction on previously unseen entities, making MLMLM a suitable approach to introducing new entities to a KB.



There are no comments yet.


page 1

page 2

page 3

page 4


STransE: a novel embedding model of entities and relationships in knowledge bases

Knowledge bases of real-world facts about entities and their relationshi...

Inductive Entity Representations from Text via Link Prediction

We present a method for learning representations of entities, that uses ...

Interpretable and Fair Comparison of Link Prediction or Entity Alignment Methods with Adjusted Mean Rank

In this work, we take a closer look at the evaluation of two families of...

A Systematic Investigation of KB-Text Embedding Alignment at Scale

Knowledge bases (KBs) and text often contain complementary knowledge: KB...

Semantic Entity Enrichment by Leveraging Multilingual Descriptions for Link Prediction

Most Knowledge Graphs (KGs) contain textual descriptions of entities in ...

Self-attention Presents Low-dimensional Knowledge Graph Embeddings for Link Prediction

Recently, link prediction problem, also known as knowledge graph complet...

SAFRAN: An interpretable, rule-based link prediction method outperforming embedding models

Neural embedding-based machine learning models have shown promise for pr...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.