Self-attention Presents Low-dimensional Knowledge Graph Embeddings for Link Prediction

12/20/2021
by   Peyman Baghershahi, et al.
0

Recently, link prediction problem, also known as knowledge graph completion, has attracted lots of researches. Even though there are few recent models tried to attain relatively good performance by embedding knowledge graphs in low dimensions, the best results of the current state-of-the-art models are earned at the cost of considerably increasing the dimensionality of embeddings. However, this causes overfitting and more importantly scalability issues in case of huge knowledge bases. Inspired by the recent advances in deep learning offered by variants of the Transformer model, because of its self-attention mechanism, in this paper we propose a model based on it to address the aforementioned limitation. In our model, self-attention is the key to applying query-dependant projections to entities and relations, and capturing the mutual information between them to gain highly expressive representations from low-dimensional embeddings. Empirical results on two standard link prediction datasets, FB15k-237 and WN18RR, demonstrate that our model achieves favorably comparable or better performance than our three best recent state-of-the-art competitors, with a significant reduction of 76.3 embeddings on average.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/22/2017

Analysis of the Impact of Negative Sampling on Link Prediction in Knowledge Graphs

Knowledge graphs are large, useful, but incomplete knowledge repositorie...
research
04/11/2021

Multiple Run Ensemble Learning withLow-Dimensional Knowledge Graph Embeddings

Among the top approaches of recent years, link prediction using knowledg...
research
11/11/2019

Decompressing Knowledge Graph Representations for Link Prediction

This paper studies the problem of predicting missing relationships betwe...
research
09/03/2022

Semi-supervised Training for Knowledge Base Graph Self-attention Networks on Link Prediction

The task of link prediction aims to solve the problem of incomplete know...
research
11/01/2019

InteractE: Improving Convolution-based Knowledge Graph Embeddings by Increasing Feature Interactions

Most existing knowledge graphs suffer from incompleteness, which can be ...
research
06/08/2021

Hyperbolic Temporal Knowledge Graph Embeddings with Relational and Time Curvatures

Knowledge Graph (KG) completion has been excessively studied with a mass...
research
03/27/2021

High-efficiency Euclidean-based Models for Low-dimensional Knowledge Graph Embeddings

Recent knowledge graph embedding (KGE) models based on hyperbolic geomet...

Please sign up or login with your details

Forgot password? Click here to reset