Log In Sign Up

Taxonomical hierarchy of canonicalized relations from multiple Knowledge Bases

by   Akshay Parekh, et al.

This work addresses two important questions pertinent to Relation Extraction (RE). First, what are all possible relations that could exist between any two given entity types? Second, how do we define an unambiguous taxonomical (is-a) hierarchy among the identified relations? To address the first question, we use three resources Wikipedia Infobox, Wikidata, and DBpedia. This study focuses on relations between person, organization and location entity types. We exploit Wikidata and DBpedia in a data-driven manner, and Wikipedia Infobox templates manually to generate lists of relations. Further, to address the second question, we canonicalize, filter, and combine the identified relations from the three resources to construct a taxonomical hierarchy. This hierarchy contains 623 canonical relations with highest contribution from Wikipedia Infobox followed by DBpedia and Wikidata. The generated relation list subsumes an average of 85 restricted.


page 1

page 2

page 3

page 4


A Hierarchical Framework for Relation Extraction with Reinforcement Learning

Most existing methods determine relation types only after all the entiti...

Weakly-Supervised Questions for Zero-Shot Relation Extraction

Zero-Shot Relation Extraction (ZRE) is the task of Relation Extraction w...

Revisiting Unsupervised Relation Extraction

Unsupervised relation extraction (URE) extracts relations between named ...

UHop: An Unrestricted-Hop Relation Extraction Framework for Knowledge-Based Question Answering

In relation extraction for knowledge-based question answering, searching...

Distantly Supervised Relation Extraction via Recursive Hierarchy-Interactive Attention and Entity-Order Perception

Distantly supervised relation extraction has drawn significant attention...

Assessing the Difficulty of Classifying ConceptNet Relations in a Multi-Label Classification Setting

Commonsense knowledge relations are crucial for advanced NLU tasks. We e...

Generating Categories for Sets of Entities

Category systems are central components of knowledge bases, as they prov...

1 Introduction

Relations mentioned in unstructured texts often share taxonomical (is-a) association with other relations. For example, in figure 1 relations father and mother shares taxonomical relation with parent. By virtue of this relation, entities Hermann and Pauline also have parent relation with the entity Albert which is also true in real-world. But such inference is hard to extract from the existing relation extraction (RE) resources as they fail to answer two following questions: first, what are all possible relations that could exist between entities? Second, how do we obtain an unambiguous taxonomical hierarchy between the identified relations?

Figure 1: With the taxonomical hierarchy, and the relations present in the sentences S1 and S2, we can infer relations shown in sentences S3 and S4.

Available RE resources show the following bottlenecks: limited relations, absence of canonical relations, and absence of hierarchy in them. The first limitation is because of pre-defined handcrafted or corpus-dependent relations list Mitchell et al. (2005); Hendrickx et al. (2009). To scale the number of relations a few datasets Riedel et al. (2010); Han et al. (2018b) use a single KB to obtain a potential list of relations. As there is no standard nomenclature and mapping followed among KBs, restriction to a single KB leads to the second bottleneck. Even though KBs like Wikidata Vrandečić and Krötzsch (2014) and DBpedia Lehmann et al. (2015) incorporate deep hierarchical ontologies, that do not explicitly address relations and extracting relation hierarchy from them is challenging.

Figure 2: Overview of hierarchy creation steps. Question 1: What are all possible relations that could exist between any two given entity types? Question 2: How do we define an unambiguous taxonomical (is-a) hierarchy among the identified relations?

Thus, it is important to create a large database of relations, that considers relation as a concept. Further, it must cover all possible relations that could exist between a pair of entities, taxonomical and semantic association between relations, and their synsets. This study initiates work in that direction. We assume properties and attributes appearing in structured knowledge bases (KBs) and ontologies are a good representative of all possible relations. Therefore, we extract relations from Wikipedia Infobox templates111 manually, and DBpedia and Wikidata in a data-driven way. We collect an exhaustive list of relations between person, organization, and location entity types. Further, we create a relation hierarchy of 623 canonical relations. We perform analyses to understand the contribution of each resource, the effects of canonicalization, the complementarity of KBs, and coverage of relations present in the existing RE datasets.

2 RE Datasets

ACE multilingual corpus Mitchell et al. (2005) is one of the most commonly used RE dataset. It arranges relations in a hierarchy of depth 1 and contains about 30 relations at leaf level. The relations at intermediate nodes are not generic enough to be scalable and also are few in numbers. Mintz et al. (2009) proposed distant supervision for automatic data generation with more number of relations using a KB. Following that Riedel et al. (2010) introduced NYT dataset with 52 relations using Freebase Bollacker et al. (2008) as a KB. Although Freebase has more than 700 properties, only 52 could qualify as relation because of the underlying corpus. Recently published datasets TACRED Zhang et al. (2017) and FewRel Han et al. (2018b), cover 42 and 100 relations respectively. Similar to our work, TACRED considers relations specific to person, organization and location entity types. FewRel contains relations from Wikidata. However, the relation count is limited in contrast to our objective.

3 Relation Hierarchy

A triple (e1,r, e2) represents relation r between head entity e1 and tail entity e2. In our relation hierarchy, we collect relations from multiple resources and group relations that share is-a relations, under the same branch. Figure 2 summarizes the following steps of hierarchy creation:

Getting relation list: Our first resource Wikipedia Infobox stores structured information into attribute-value pairs following an Infobox template. We manually selected 170, 77 and 89 Infobox templates for person, for organization, and location respectively. Due to noisy nature of Infobox templates, we manually scan each of these template pages to curate a list of relations. We refer this list as .

For the other two resources DBpedia and Wikidata, we follow a data-driven approach. We parse Wikidata json dump222 and DBpedia mapping-based Infobox dump333 for generating triples. Then we collect all the unique relations from the triples dataset (where e1 and e2 is one of the three type person, organization, and location). The two lists of relations are henceforth referred as (from DBpedia) and (from Wikidata).

Name canonicalization: Even though the three sources are closely related, they follow different nomenclature for their relations. Thus, same relation can have different names in different lists. For example, consider relation placeOfBirth, it is birth_place in Wikipedia Infobox , birthPlace in DBpedia, and place of birth in wikidata, .

To canonicalize relation names, we follow current policy of DBpedia. For example, if a relation name is a single word, consider it as it is, given all the characters are in small-case. Otherwise, capitalize all the words except the first word, remove in-between spaces and concatenate all the words. For example, place of birth becomes placeOfBirth. In case of multiple names for the same relation (as in the earlier example), we choose one of them and store the respective mapping. Following this procedure, we obtain canonicalized relation lists , and from , and repectively.

Filtering: This step ensures that our relation hierarchy focuses on frequently occuring relations. We filter out relations from the list if they appear in less than 100 infoboxes. Similarly, a relation is filtered out from the lists and if it has less than 100 associated triples.

Hierarchy creation: A relation r

describes relationship between two entities. Thus, it is natural to classify based on the head and tail entity types at the top level. Consider a relation

founder, head entity type is organization (org) and tail entity is of type person (per), thus it falls under branch org-per. Since it is organization specific relation, org-per.founder will fall under org which falls under the root rel.

Initial levels of hierarchy are described as:

  • At depth 0: root node referred as rel

  • At depth 1: we distinguish based on head entity type. In this level there are 3 nodes per (person relations), loc (location relations), and org (organization relations).

  • At depth 2: we distinguish based on both head and tail entities. In this level, there are 9 nodes (For example, per-loc head entity: person and tail entity: location) and each node of this level are henceforth referred as a bucket for relations.

All the relations from the three filtered lists are distributed across 9 buckets. We manually arrange relations in the hierarchy whenever is-a association exists between two relations.

Taxonomically similar relations (child nodes) are placed under the same parent node. If parent node is not present in the filtered relation list, canonical relation list is referred. If present, that referred relation is chosen. Otherwise, a new parent node is introduced. In our hierarchy, we have introduced a total of 12 new nodes.

Hierarchy merging: Following guidelines in previous step, hierarchies , and are created. Finally, they are merged into one common hierarchy by eliminating the duplicates and placing taxonomically similar relation under the same branch.

4 Analysis

Table 1 shows basic statistics of three individual hierarchies , , , and the common hierarchy . All hierarchies have maximum depth of 5 (6 levels). All relations from the filtered lists are distributed at depths 3, 4 and 5. Distribution of relations at depth 4 and 5 gives more fine-grained information about relations shared between two entities. In the common hierarchy, loc-loc bucket has the most number of relations (113) whereas org-loc bucket has the least number (24).

@d = 3
@d = 4
@d = 5
623 357 247 19
351 177 168 6
282 162 110 10
267 209 52 6
Table 1: Relation Count of hierarchies and number of relations at various depths (Relation @ d =).

Effects of canonicalization: Relation name canonicalization has played an important role in eliminating redundant relations from (Table 2). This in turn helped significantly in finding common relations among the resources. Since DBpedia and Wikidata are structured at its core, canonicalization has not affected much.

Person Organization Location
Infobox 660 154 228 165 183 84
Dbpedia 94 92 103 99 91 86
Wikidata 71 71 73 72 98 97
Table 2: Relation counts before (B) and after (A) canonicalization.

Coomplementarity of resources: Figure 3 shows the contribution of resources towards relation buckets. Manually collected relations from Wikipedia Infobox dominate 7 out of the 9 buckets. The contributions of DBpedia and Wikidata towards each bucket is almost similar.

Figure 3: Distribution of relations from different resources in each of the 9 buckets.

Figure 4 shows contribution of each resource towards the common hierarchy. Only about relations are common among the three resources. This analysis indicates complementarity of the resources.

Figure 4: Contribution of resources towards common hierarchy.

Comparison with relation list of RE datasets: The main objective behind this study was to highlight major bottlenecks of RE datasets (sec 1). Table 3 briefly shows how relations from RE datasets get subsumed in our relation hierarchy. Our hierarchy covers an average of 62% of relations when all the relation of a dataset is considered and 85.35% of relations when relation’s head and tail entity types are restricted to person, organization, and location types.

RE Dataset relation count depth relation subsumed
ACE 2004 24 (17) 1 11
NYT2010 Dataset 51 (47) 0 35
TACRED 41 (29) 0 27
FewRel 100 (64) 0 61
Relation Hierarchy 623 5 (3.42) -
Table 3: RE Datasets with relation count(relation with head and tail entity of type person, organization, and location), hierarchy depth, and numbers of relation subsumed in Relation Hierarchy

5 Use Cases and Applications

RE dataset creation: We provide a comprehensive list of 623 relations. This list, along with relation hierarchy, can further be used as a set of model relations for creating a sizeable sentence-level dataset for RE.
Improving RE training data: Introduction of relation hierarchy will also guarantee training data for intermediate nodes. This will help solve the problem for some of the long-tail labels. For example, in TACRED, percentage of training instances for city_of_death, country_of_death, and stateorprovince_of_death are 0.12%, 0.01% and 0.07% respectively which is significantly less than average, 0.46 % (excluding no_relation). In the relation hierarchy, these relations will be placed under place_of_death. This new relation will contain instances of all the three (combined percentage 0.20%).
Hierarchical RE Models: Han et al. (2018a) proposed a hierarchical RE model on NYT2010 dataset. Their hierarchical model significantly outperformed other baselines by utilizing Freebase hierarchy. Our relation hierarchy being more comprehensive and scalable, we expect better learning for hierarchical RE models.
Knowledge Graph Completion: Despite the large size of knowledge bases, they are far from complete. For example, only 25% of Dbpedia person and 12% of Wikidata person has placeOfBirth information, and 27% of Dbpedia location and 24% of Wikidata location has country information. Once we map relations from multiple resources to the canonicalized relations of relation hierarchy, we can easily compare the triples of different knowledge resources.

6 Conclusion and Future Work

This study explored more than 1500 properties and attributes from Wikipedia Infoboxes, DBpedia, and Wikidata to generate lists of prospective relations. These relations were used to create a hierarchy of 623 canonical relations. Our analysis indicates only 10% overlap among the three resources. Additionally, our relation hierarchy subsumes 85% of relations from RE datasets with restricted entity types. In future work, we aim to extend this relation hierarchy by including more entity types, and more resources like YAGO Suchanek et al. (2007) and schema.org444 in an automated manner. Furthermore, we also intend to use this extensive list of relations along with the relation hierarchy for generating a large-scale dataset for fine-grained RE.


  • K. Bollacker, C. Evans, P. Paritosh, T. Sturge, and J. Taylor (2008) Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pp. 1247–1250. Cited by: §2.
  • X. Han, P. Yu, Z. Liu, M. Sun, and P. Li (2018a) Hierarchical relation extraction with coarse-to-fine grained attention. In

    Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing

    pp. 2236–2245. Cited by: §5.
  • X. Han, H. Zhu, P. Yu, Z. Wang, Y. Yao, Z. Liu, and M. Sun (2018b) FewRel:a large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation. In EMNLP, Cited by: §1, §2.
  • I. Hendrickx, S. N. Kim, Z. Kozareva, P. Nakov, D. Ó Séaghdha, S. Padó, M. Pennacchiotti, L. Romano, and S. Szpakowicz (2009)

    Semeval-2010 task 8: multi-way classification of semantic relations between pairs of nominals. In Proceedings of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions, pp. 94–99. Cited by: §1.
  • J. Lehmann, R. Isele, M. Jakob, A. Jentzsch, D. Kontokostas, P. N. Mendes, S. Hellmann, M. Morsey, P. Van Kleef, S. Auer, et al. (2015) DBpedia–a large-scale, multilingual knowledge base extracted from wikipedia. Semantic Web 6 (2), pp. 167–195. Cited by: §1.
  • M. Mintz, S. Bills, R. Snow, and D. Jurafsky (2009) Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2-Volume 2, pp. 1003–1011. Cited by: §2.
  • A. Mitchell, S. Strassel, S. Huang, and R. Zakhary (2005) Ace 2004 multilingual training corpus. Linguistic Data Consortium, Philadelphia 1, pp. 1–1. Cited by: §1, §2.
  • S. Riedel, L. Yao, and A. McCallum (2010) Modeling relations and their mentions without labeled text. In

    Joint European Conference on Machine Learning and Knowledge Discovery in Databases

    pp. 148–163. Cited by: §1, §2.
  • F. M. Suchanek, G. Kasneci, and G. Weikum (2007) Yago: a core of semantic knowledge. In Proceedings of the 16th international conference on World Wide Web, pp. 697–706. Cited by: §6.
  • D. Vrandečić and M. Krötzsch (2014) Wikidata: a free collaborative knowledge base. Cited by: §1.
  • Y. Zhang, V. Zhong, D. Chen, G. Angeli, and C. D. Manning (2017) Position-aware attention and supervised data improve slot filling. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP 2017), pp. 35–45. External Links: Link Cited by: §2.