Multiple Run Ensemble Learning withLow-Dimensional Knowledge Graph Embeddings

04/11/2021
by   Chengjin Xu, et al.
0

Among the top approaches of recent years, link prediction using knowledge graph embedding (KGE) models has gained significant attention for knowledge graph completion. Various embedding models have been proposed so far, among which, some recent KGE models obtain state-of-the-art performance on link prediction tasks by using embeddings with a high dimension (e.g. 1000) which accelerate the costs of training and evaluation considering the large scale of KGs. In this paper, we propose a simple but effective performance boosting strategy for KGE models by using multiple low dimensions in different repetition rounds of the same model. For example, instead of training a model one time with a large embedding size of 1200, we repeat the training of the model 6 times in parallel with an embedding size of 200 and then combine the 6 separate models for testing while the overall numbers of adjustable parameters are same (6*200=1200) and the total memory footprint remains the same. We show that our approach enables different models to better cope with their expressiveness issues on modeling various graph patterns such as symmetric, 1-n, n-1 and n-n. In order to justify our findings, we conduct experiments on various KGE models. Experimental results on standard benchmark datasets, namely FB15K, FB15K-237 and WN18RR, show that multiple low-dimensional models of the same kind outperform the corresponding single high-dimensional models on link prediction in a certain range and have advantages in training efficiency by using parallel training while the overall numbers of adjustable parameters are same.

READ FULL TEXT
research
10/14/2020

Multi-teacher Knowledge Distillation for Knowledge Graph Completion

Link prediction based on knowledge graph embedding (KGE) aims to predict...
research
01/08/2022

Scaling Knowledge Graph Embedding Models

Developing scalable solutions for training Graph Neural Networks (GNNs) ...
research
12/21/2018

COSINE: Compressive Network Embedding on Large-scale Information Networks

There is recently a surge in approaches that learn low-dimensional embed...
research
12/20/2021

Self-attention Presents Low-dimensional Knowledge Graph Embeddings for Link Prediction

Recently, link prediction problem, also known as knowledge graph complet...
research
11/05/2020

Runtime Performances Benchmark for Knowledge Graph Embedding Methods

This paper wants to focus on providing a characterization of the runtime...
research
07/31/2023

Towards Semantically Enriched Embeddings for Knowledge Graph Completion

Embedding based Knowledge Graph (KG) Completion has gained much attentio...
research
01/03/2022

Swift and Sure: Hardness-aware Contrastive Learning for Low-dimensional Knowledge Graph Embeddings

Knowledge graph embedding (KGE) has drawn great attention due to its pot...

Please sign up or login with your details

Forgot password? Click here to reset