HittER: Hierarchical Transformers for Knowledge Graph Embeddings

08/28/2020
by   Sanxing Chen, et al.
0

This paper examines the challenging problem of learning representations of entities and relations in a complex multi-relational knowledge graph. We propose HittER, a Hierarchical Transformer model to jointly learn Entity-relation composition and Relational contextualization based on a source entity's neighborhood. Our proposed model consists of two different Transformer blocks: the bottom block extracts features of each entity-relation pair in the local neighborhood of the source entity and the top block aggregates the relational information from the outputs of the bottom block. We further design a masked entity prediction task to balance information from the relational context and the source entity itself. Evaluated on the task of link prediction, our approach achieves new state-of-the-art results on two standard benchmark datasets FB15K-237 and WN18RR.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/29/2023

Representation Learning on Hyper-Relational and Numeric Knowledge Graphs with Transformers

A hyper-relational knowledge graph has been recently studied where a tri...
research
04/22/2020

TransEdge: Translating Relation-contextualized Embeddings for Knowledge Graphs

Learning knowledge graph (KG) embeddings has received increasing attenti...
research
03/28/2023

Pre-training Transformers for Knowledge Graph Completion

Learning transferable representation of knowledge graphs (KGs) is challe...
research
05/22/2022

Relphormer: Relational Graph Transformer for Knowledge Graph Representation

Transformers have achieved remarkable performance in widespread fields, ...
research
09/15/2015

Macau: Scalable Bayesian Multi-relational Factorization with Side Information using MCMC

We propose Macau, a powerful and flexible Bayesian factorization method ...
research
05/18/2016

Leveraging Lexical Resources for Learning Entity Embeddings in Multi-Relational Data

Recent work in learning vector-space embeddings for multi-relational dat...
research
09/15/2023

Research on Joint Representation Learning Methods for Entity Neighborhood Information and Description Information

To address the issue of poor embedding performance in the knowledge grap...

Please sign up or login with your details

Forgot password? Click here to reset