SCEF: A Support-Confidence-aware Embedding Framework for Knowledge Graph Refinement

02/18/2019 ∙ by Yu Zhao, et al. ∙ Southwestern University of Finance and Economics 0

Knowledge graph (KG) refinement mainly aims at KG completion and correction (i.e., error detection). However, most conventional KG embedding models only focus on KG completion with an unreasonable assumption that all facts in KG hold without noises, ignoring error detection which also should be significant and essential for KG refinement.In this paper, we propose a novel support-confidence-aware KG embedding framework (SCEF), which implements KG completion and correction simultaneously by learning knowledge representations with both triple support and triple confidence. Specifically, we build model energy function by incorporating conventional translation-based model with support and confidence. To make our triple support-confidence more sufficient and robust, we not only consider the internal structural information in KG, studying the approximate relation entailment as triple confidence constraints, but also the external textual evidence, proposing two kinds of triple supports with entity types and descriptions respectively.Through extensive experiments on real-world datasets, we demonstrate SCEF's effectiveness.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1. Introduction

Knowledge graphs (KGs), which provide effective well-structured relational information between entities, have proven useful for many knowledge-driven AI and NLP tasks (Berant et al., 2013; Bordes et al., 2014; Zhong et al., 2015). A typical KG usually consists of a huge amount of knowledge triples in the form of (head entity, relation, tail entity), or the abbreviation . The past decade has witnessed great thrive in building web-scale KGs, e.g., Freebase (Bollacker et al., 2008), Google Knowledge Graph (Dong et al., 2014) and other domain-specific KGs. However, these knowledge graphs cannot reasonably reach full coverage and always suffer from incompleteness due to a large volume increasing and even infinite real-world knowledge facts (Paulheim, 2017). Accordingly, knowledge graph construction and completion are significant for KG-driven tasks. Recently, open information extraction (Open IE) (Banko et al., 2007; Stanovsky et al., 2007), automatic neural relation extraction (Lin et al., 2016) and crowd-sourcing mechanism are widely used for KG construction, while these approaches unfortunately may bring noises in KG due to insufficient human supervision(Liang et al., 2017; Heindorf et al., 2016). For instance, recent open IE model on benchmark achieves only 24% precision when the recall is 67% (Stanovsky et al., 2007).

To address those shortcomings, in this paper we focus on KG refinement (Paulheim, 2017), attempting to accomplish two targets simultaneously: (a) improving the coverage of KG, i.e., KG completion, by adding additional facts to KG, and (b) improving the correctness of KG, i.e., KG correction, by identifying and removing errors. KG refinement is critical to improve knowledge graph which is a backbone of many knowledge-driven intelligent systems (Berant et al., 2013; Bordes et al., 2014). In recent years, various embedding models (Bordes et al., 2013; Guo et al., 2015; Jenatton et al., 2012; Ji et al., 2015; Lin et al., 2015b; Nickel et al., 2016b, 2011; Shi and Weninger, 2017; Wang et al., 2014; Socher et al., 2013; Trouillon et al., 2016; Xiao et al., 2016) have been proposed for KG completion but with inappropriate assumption that no noise exists in KG, ignoring error detection which should be significant and essential for KG refinement as well (Paulheim, 2017).

In this paper, to refine KG, we propose a novel support-confidence-aware KG embedding framework (SCEF), which considers triple quality as well in contrast to conventional methods. We implement KG completion and correction simultaneously by learning knowledge representations with triple support and confidence. The framework with support-confidence has been widely studied in research field such as data mining (Mitra et al., 2002). Fig. 1 demonstrates a brief illustration of our support-confidence-aware framework, where KG suffers from incompleteness and noises after automatic KG construction. KG is expected to be improved via KG refinement by learning embeddings with our proposed SCEF model.

Figure 1. Automatic KG construction and KG refinement by learning embeddings.

Specifically, SCEF follows the promising translation-based framework first proposed by (Bordes et al., 2013)

, and builds final energy function by combining conventional translation-based model with support and confidence. We propose two triple supports considering two typical external textual evidence: entity types and entity descriptions respectively, which correspondingly provide rich pragmatic and semantic information for triple support estimation

(Xie et al., 2016a; Xiao et al., 2017). Moreover, we further propose triple confidence constraints by studying the approximate relation entailment (Ding et al., 2018) according to internal structural information in KG, beyond the local and global confidence proposed by (Xie et al., 2018).

We evaluate our models on three tasks including KG noise detection, KG completion and triple classification. Experimental results demonstrate that our proposed model outperform the baseline on all tasks, which confirms the capability of SCEF in KG refinement. The main contributions of this work are concluded as follows:

  • We propose a novel support-confidence-aware KG embedding framework for knowledge graph refinement, which use both internal structural information in KG and external textual evidence.

  • We evaluate our SCEF models on several datasets: FB15K-N1, FB15K-N2 and FB15K-N3, which have different noise rates extended from FB15K (Bordes et al., 2013), and outperforms previous models on all tasks.

  • The inspiring idea of support-confidence embedding framework would not only boost the research on knowledge graph refinement, but also the relational machine learning

    (Nickel et al., 2016a), network embedding (Tang et al., 2015).

2. Related Work

2.1. Knowledge Graph Refinement

Knowledge graph refinement (KGR) is essential after automatic KG construction (Dong et al., 2014), since the result may never be perfect whichever approach is taken for constructing knowledge graph (Paulheim, 2017; Melo and Paulheim, 2017; Heindorf et al., 2016). Various methods for KGR have been proposed (Paulheim, 2017), which can differ along three distinct orthogonal dimensions: (i) the overall goal of the method, i.e., completion (Bordes et al., 2013; Socher et al., 2013) vs. correction (Melo and Paulheim, 2017; Heindorf et al., 2016) of KG; (ii) the refinement target (e.g., relations between entities (Socher et al., 2013), entity types (Nickel et al., 2012)), and (iii) the data used by the approach (i.e., only KG itself (Bordes et al., 2013), or further external sources (Xie et al., 2016b; Xie et al., 2016a)). However, most conventional approaches are only used for one goal as yet, while a combination between completion and error detection methods could be of great value (Paulheim, 2017). Jia et al. (2018)

propose a crisscrossing neural network for KG completion and correction at the same time, while have high complexity and computational cost. In this paper, we concentrate on KG completion and correction simultaneously, and introduce two triple supports and a triple confidence constraints for KGR, by considering the typical external heterogeneous source (i.e., entity hierarchical types and descriptions) beyond the KG itself.

2.2. Knowledge Graph Embedding

Recent years knowledge graph embedding (see more in this survey (Wang et al., 2017)

) has become a hot research topic. The key idea is to encode all the entities and relations in KG into a latent semantic vector space, so as to simplify manipulation without losing KG’s inherent structural information. Various embedding methods have been proposed in recent years, which can be roughly classified into two categories according to the information they used: (i) those which learn embeddings only with KG at hand, e.g., TransE

(Bordes et al., 2013) and its extensions (Wang et al., 2014; Lin et al., 2015b; Ji et al., 2015; Xiao et al., 2016), the RESCAL (Nickel et al., 2011) and its extensions (Liu et al., 2017), and the neural network models (Socher et al., 2013; Dettmers et al., 2017); (ii) those learning embeddings by combining existing KG with external heterogeneous information, e.g., entity hierarchical types (Xie et al., 2016a; Chang et al., 2014), entity descriptions (Xiao et al., 2017; Xie et al., 2016b; Zhong et al., 2015), plain text(Zhang et al., 2015), and relation paths (Lin et al., 2015a). However, all these methods assume that all the facts in KG hold without noise, which is unreasonable especially for KGs constructed automatically without sufficient human supervision. Recently, Xie et al. (2018) propose a embedding method (CKRL) with confidence to deal with this issue, which is a pioneer work within KG embedding framework. But it ignores the rich semantic information in external textual information which is strong evidence to judge triple quality (Xiao et al., 2017; Xie et al., 2016b; Xie et al., 2016a), and also very prone to overfitting since both the triple score and the confidence merely derive from internal structural information in KG (Wang et al., 2017). To the best of our knowledge, our proposed model is the first knowledge graph embedding method in the second category to consider both the triple support and confidence for KG refinement. In this paper, we concentrate on refining noisy KGs on the basis of the promising translation-based knowledge graph embedding model (TransE), which is not difficult to be replaced with other enhanced KG embedding model (Wang et al., 2014; Lin et al., 2015b).

3. Methodology

We first give the notations used in this paper. For each triple , the head and tail entities and the relation , where and represent the sets of relations. and represent the hierarchical types of head and tail respectively. and denote the descriptions of them respectively. stand for the overall training set with noises.

3.1. Support-Confidence-aware KG Embedding Framework

To refine knowledge graph (i.e., KG completion and error detection), we first introduce two promising concepts: triple support and triple confidence for each triple fact. Triple support, derived from the external textural evidence, reflects the reliability of a triple. Triple confidence, measured with internal structural information, describes the correctness of a triple. The fact with higher support and confidence possesses higher quality, and should be more considered reasonably. We set the triple model as the same of TransE (Bordes et al., 2013): , which represent the dissimilarity score between head and tail with relation under translation assumption. Here, the support-confidence-aware KG embedding energy function (see Figure 2) is correspondingly designed as follows:

(1)

stands for the triple support, which is showed in detail in Section 3.2. A higher triple support implies that the corresponding triple is more reliable. represents the triple confidence. A higher triple confidence indicates the triple is more credible.

Figure 2. The support-confidence-aware KG embedding energy function.

The support-confidence framework is generic and is not limited to a specific model of triple, which can be used for improving other knowledge graph embedding models. CKRL, proposed by Xie et al. (2018), can be taken as a special case of our SCEF if is set to 1, defectively ignoring the rich semantic information in external textural evidence. Actually triple quality can be labeled by human-being experts, but this work is much time-consuming and subjective (Paulheim and Bizer, 2014). In next section, we introduce automatic methods to estimate triplet support and confidence according to KG and external textual evidence respectively. In SCEF, we bring in triple support and confidence to learn more about the significant facts, and thus could obtain better knowledge embeddings for KG refinement.

Figure 3. The example of triple, with entity hierarchical types and entity descriptions.

3.2. Triple Support with Textual Evidence

3.2.1. Triple Support with Entity Types

We first utilize the entity hierarchical types for the triple support (). Entity hierarchical types information implies different roles an entity may play in different scenarios (Chang et al., 2014). Most typical knowledge graph (e.g. Freebase(Bollacker et al., 2008), DBpedia (Lehmann et al., 2015)) have entity type information. Entity types usually consist of hierarchical structures, in which the lower granularity of semantic concepts are considered as the sub-type of entities. Generally most entities possess more than one hierarchical type. For instance, in Fig. 3, State of Hawaii has a variety of types (e.g. people/place_of_born, areas/sovereign_state and areas/Administrative_area) and shows different attributes under different types.

The head and tail entity hierarchical types are strong evidence to estimate the triple support. For instance, the incomplete triple (?, was_born_in, State of Hawaii), in which the head suppose to be filled with a living things (Type:people/person), is more creditable than which is filled with non-living things (Type: book/written_work). In other words, although both the triples (Donald Trump, was_born_in, State of Hawaii) and (Pride and Prejudice, was_born_in, State of Hawaii) are not true, but we still believe that the support of the former is higher than the support of the latter due to their distinct types, i.e., the type of Donald Trump (people/person) is more reasonable. Here, we build the entity type triple, by replacing both head and tail with their corresponding hierarchical types: , to estimate the triple support.

To build triple support with hierarchical types, inspired by TKRL (Xie et al., 2016a), we first encode the hierarchical type information into representation learning with a general form. Generally most entities in KGs have more than one hierarchical type. The general form of type encoder, in which the type representation of entity will be the weighted summation of all type representations, is as follow:

(2)

where is the number of types of entity , is the -th type of entity , and is the representation and corresponding weight for respectively. Secondly, we utilize the promising weighted hierarchical embedding (WHE) method (Xie et al., 2016a), considering that different granularities of sub-type in hierarchical structures may vary in significance in type representation, to build the representation of as follows:

(3)

in which is the number of layers in the hierarchical structure, is the sub-type representation of , is the corresponding weight of . The more details can be find in (Xie et al., 2016a).

Finally, we design a novel model of the triple support with entity hierarchical types. As we discussed above, following translation-assumption framework (Bordes et al., 2013), we build the distance of the entity type triple: . and stand for the representation of head type and tail type respectively which are calculated by WHE. The triple support with hierarchical types is designed as follows:

(4)

where is the sigmoid or logistic function to convert the type triple score into triple support. A higher the

implies that the triple is more probable to hold.

3.2.2. Triple support with Entity Descriptions

TS would fail to work if the types of head and tail exactly match but the fact is actually false, such as (Donald Trump, was_born_in, State of Hawaii). However, the textual descriptions can discover semantic relevance and offer precise semantic expression (Xiao et al., 2017). The semantic relevance between entities is capable to recognize the true triples, and precise semantic expression could promote the discriminative ability between two triples. Here, we design the entity description triple to estimate the triple support, by replacing both head and tail with their corresponding descriptions: . In the following, we introduce an novel approach to build triple support with entity descriptions (DS).

From each shot description, we generate a set of keywords which is capable of capturing the main ideas of entities. The assumption is that similar entities should have similar descriptions, and correspondingly have similar keywords. Those triple support may be detected in the internal contact of their keywords. We formulate entity descriptions as . is the set of keywords in entity description. is the size of words set. We take advantage of conventional neural network (CNN) (Lai et al., 2015; Xie et al., 2016b) to model entity description . The CNN model can takes word orders, i.e., complicated local interactions of keywords in entity description, into consideration.

The -th output vector of convolution layer is calculated as:

(5)

where is the convolution kernel for all input vectors of -th convolution layer after window process and is the optional bias.

is the activation function such as

tanh or ReLU. is the -th vector of which is obtained by concatenating column vectors in -th window of the polling output of (-1)-th layer. The pooling process shrinks the parameter space of CNN and filter noises after every convolution layer. We use

-max-pooling and mean-pooling strategies respectively in different pooling layers. After the last pooling layer, we obtain the representation of entity description

.

Finally, we design a novel model of the triple support with entity descriptions. Under translation-assumption, we build the distance of the entity description triple: . and stand for the representation of head type and tail descriptions respectively which are calculated by CNN. The triple support with entity descriptions is designed as follows:

(6)

A higher the implies that the triple is more probable to hold.

3.2.3. Overall Support Model

The overall triplet support combines with two kinds of support stated above. We have:

(7)

where is hyper-parameter.

3.3. Triple Confidence with KG

3.3.1. Ckrl

As we introduced in Section 2, CKRL, proposed by Xie et al. (2018), is a promising model to estimate the triple confidence with the internal structural information in KG. Inspired by their work, we take advantage of both the local triple confidence and global path confidence (see more in (Xie et al., 2018)). Furthermore, we also study approximate relation entailment constraints (EC) from internal KG for triple confidence estimation.

3.3.2. Approximate Relation Entailment as Confidence Constraint

Approximate relation entailment means that an ordered pair of relations that the former approximately entails the latter, e.g.

was_born_in and nationality, stating that a person born in a country is very likely, but unnecessarily, to have a nationality of that country. Each such relation pair () is associated with a weight () to indicate the confidence level of entailment: . A higher weight stands for a higher level of confidence. Following Wang et al. (2017), the relation entailments can be derived automatically from KG by modern rule mining systems (Galaarraga et al., 2015). Let denotes the set of all approximate relation entailments. Therefore, we can denote the approximate entailment between relations and with confidence level as follows:

3.4. Objective Formalization and Optimization

We introduce the training objective of our model. Following TransE (Bordes et al., 2013), we use margin-based optimization criterion to train our model. The main idea is that the model scoring function value of true knowledge triple in training set should be lower than the corrupt one, the head or tail of which is replaced by a random one. Note that we do not replace both head and tail with random one at the same time. A triple will not be considered as a negative sample if it is already in training set

. To learn embeddings, we minimize the hinge loss function

as follows:

where is the dissimilarity score of positive triple and is that of negative triple, and

is the margin hyperparameter. The triple support

is determined by (7). represents the negative triple set.

It is not absolutely necessary to use hinge loss function (Zhao et al., [n. d.]). However, it is very common to use hinge loss for learning embedding (like TransE, NTN, etc) just as our model did. The embeddings of all entities, relations, sub-types and keywords are denoted as

respectively initialized randomly. We use mini-batch stochastic gradient descent (SGD) for optimization.

As point out by (Brian Murphy and Mitchell, 2012), it would be uneconomical to save all negative properties of an entity or a concept. Therefore, we further require entities to have non-negative vectorial representations(Ding et al., 2018). In most cases, non-negative will further induce sparsity and interpretability (Lee and Seung, 1999). We perform the following procedure iteratively for a given number of iterations. First, we sample a small set (minibatch) of triples from the training set , and then for each positive triple in it, we construct a negative sample by replace the head or tail with random one. The parameters are then updated by taking a gradient descent step gradually.

Figure 4. Evaluation results on KG noise detection.

4. Experiment

4.1. Datasets

Our experiments are conducted on a public benchmark dataset, FB15K (Bordes et al., 2013) that is a typical knowledge graph extracted from Freebase (Bollacker et al., 2008), and three extended datasets (FB15K-N1, FB15K-N2 and FB15K-N3) which is generated based on FB15K with different noise rates (i.e., 10%, 20% and 40% respectively) to simulate the real-world KG construction with errors (Xie et al., 2018).

Given a positive triple in KG, the head or tail is randomly replaced to form a negative one or . In order to generate harder and more confusing noises, (or ) should have appeared in the head (or tail) position with the same relation, which means that the tail entity of relation was_born_of in negative triples should also be a place. All three noisy datasets share same entities, relations, validation and test sets with FB15K, will all generated negative triples fused into the original training set of FB15K. The statistics are listed in Table 1.

Following (Galaarraga et al., 2015), we use AMIE+111https://www.mpi-inf.mpg.de/departments/databases-and-information-systems/research/yago-nage/amie to extract approximate relation entailment automatically from FB15K. As suggested by Guo et al. (2018), we only consider entailments with PCA confidence222It denotes the confidence under the partial completeness assumption, see more in (Galaarraga et al., 2015). higher than 0.8. As such, 535 approximate relation entailments are extracted from FB15K.333For instance, /people/place_of_birth /location/people_born_here; /film/directed_by /director/film

Dataset #Entities #Rel.s #Train #Valid #Test
FB15k 14,951 1,345 483,142 50,000 59,071
Datasets FB15k-N1 FB15k-N2 FB15k-N3
#Neg triples 46,408 93,782 187,925
Table 1. Statistics of the datasets.

4.2. Experimental Settings

In experiment, we evaluate our SCEF model with three different combination strategies. SCEF (EC) represents the strategy which only considers triple confidence with relation entailment constraints. SCEF (EC+TS) considers both confidence constraints and type support, while SCEF(EC+TS+DS) considers confidence constraints and two kinds of triplet support. We choose TransE (Bordes et al., 2013) and CKRL (Xie et al., 2018) as baseline for comparison. As the datasets are the same, we directly reprint the experimental results of several baselines from the literature. We train our SCEF model using mini-batch SGD. The margin is empirically set to 1. We select the learning rate in the stochastic gradient descent among {0.0001, 0.001, 0.01}, the dimension of entity, relation, entity type and keyword embedding in all models in a range of {50, 100} on the validation set. For overall triple support model, the hyperparameter is set as 0.5. For triple confidence, we set the parameter of CNN are: #window size=2, #convolution layer = 2, #dimension of feature map = .

4.3. Knowledge Graph Noise Detection

This task aims to detect possible noises in knowledge graphs according to their triple scores, in order to verifying the capability of our SCEF models in identifying noises in KGs. We utilize translation-assumption method TransE: M= as our triple model. Following the triple classification protocol in (Socher et al., 2013), we rank all triples in training set with their model score. Therefore, the higher the model score, the more likely the triple is noise.

Results: Fig. 4 shows the evaluation results (precision/recall curves) of KG noise detection, from which we can observe that:

  • Our proposed models outperform all the baselines on all noisy datasets, which confirms the capability of our SCEF models in error detection for KG refinement.

  • SCEF (EC+TS+DS) has impressive improvement in error detection compare to other support-confidence-aware methods. It indicate that the triple support with entity descriptions can provide significant help for error detection.

  • SCEF (EC+TS) achieves better performance than SCEF (EC). It indicates that the triple support with entity type can further boost KG correction beyond the triple confidence.

  • SCEF (EC) performs better than CKRL (LT+PP+AP), which implies that the approximate relation entailment constraints can improve the learning of triple confidence, so as to help KG noise detection.

DATASET FB15K-N1 FB15K-N2 FB15K-N3
METRICS Mean Rank Hits@10(%) Mean Rank Hits@10(%) Mean Rank Hits@10(%)
Raw Filter Raw Filter Raw Filter Raw Filter Raw Filter Raw Filter
TransE 240 144 44.9 59.8 250 155 42.8 56.3 265 171 40.2 51.8
CKRL (LT) 237 140 45.5 61.8 243 146 44.3 59.3 244 148 42.7 56.9
CKRL (LT+PP) 236 139 45.3 61.6 241 144 44.2 59.4 245 149 42.8 56.8
CKRL (LT+PP+AP) 236 138 45.3 61.6 240 144 44.2 59.3 245 150 42.8 56.6
SCEF (EC) 235 137 45.8 61.9 238 143 44.8 60.2 243 147 43.1 57.2
SCEF (EC+TS) 232 136 46.2 62.2 236 140 45.1 60.7 240 145 44.1 58.2
SCEF (EC+TS+DS) 231 136 46.2 62.8 235 140 45.3 60.9 240 144 44.2 58.2
Table 2. Evaluation results on entity prediction. (The lower the better for Mean Rank, whereas Hits@10(%) is on the contrary.)

4.4. Knowledge Graph Completion

The classical knowledge base completion task aims to complete a triple when one of its head, tail or relation is missing, i.e., to predict how likely some additional facts (triples) are held. Here, we only focus on entity prediction, determined by TransE(Bordes et al., 2013): . We conduct two typical measures proposed by (Bordes et al., 2013): Mean Rank and Hits@10, and the different evaluation settings of “Raw” and “Filter” (See (Bordes et al., 2013) for details).

Results: Table 2 shows the results of entity prediction with different noise rates, from which we observe that:

  • All support-confidence-aware SCEF models achieve better performance comparing with baseline on all noisy datasets, which confirms the capability of our models in KG completion beyond KG error detection.

  • Our methods achieve more significant improvement as the noise rate in KGs increases, comparing with evaluation results between the three noisy datasets. It verifies that considering the support-confidence in KG embedding is very essential especially when KGs have high rate of noises.

  • Both SCEF (EC+TS) and SCEF (EC+TS+DS) perform better than SCEF (EC). It demonstrates that the external information (i.e., entity type and description) could further benefit KG completion.

Dataset
FB15K-
N1
FB15K-
N2
FB15K-
N3
TransE 81.3 79.4 76.9
CKRL(LT) 81.8 80.2 78.3
CKRL(LT+PP) 81.9 80.1 78.4
CKRL(LT+PP+AP) 81.7 80.2 78.3
SCEF (EC) 82.1 80.9 79.2
SCEF (EC+TS) 82.4 81.0 80.3
SCEF (EC+TS+DS) 82.6 81.2 80.3
Table 3. Evaluation results on triple classification

4.5. Triple Classification

Triple classification aims to predict correct facts in the test data, which could be viewed as a binary classification problem. Following the same protocol in (Socher et al., 2013), we use validate set to find a threshold for each relation such that if , the triple will be classified to be positive, otherwise to be negative. Since there are no explicit negative triples in existing knowledge graphs, we construct negative triples in validation and test set following the same protocol in (Socher et al., 2013), with equal number of positive and negative examples. The final accuracy is based on how many triplets are classified correctly.

Results: Table 3 shows the resulting accuracy of each model. We can find that:

  • The SCEF models outperform baseline on all datasets, which prove that support-confidence-aware framework can be helpful for relation triple classification as well.

  • More specifically, SCEF (EC+TS+DS) model improves 0.9%, 1.0% and 2.0% on FB15K-N1, FB15K-N2 and FB15K-N3 respectively, it reaffirms that our method become more significant with higher noise rates.

5. Conclusion and Future Work

Knowledge graph refinement (e.g., completion and correction), is an important but underexplored problem. In this paper, we propose SCEF, a novel support-confidence-aware framework for KG completion and correction simultaneously by learning embeddings with triple support and confidence. We not only consider the internal structural information in KG, studying the approximate relation entailment as triple confidence constraints, but also the external textual evidence, proposing two kinds of triple supports with entity types and descriptions respectively. Through extensive experiments on three real-world datasets, we demonstrate SCEF’s effectiveness over state-of-the-art baselines.

We will explore the following research directions in future: (i) more external source are expected to improve the triple quality. (ii) apply the support-confidence framework to improve network embedding which faces noise issue as well.

Acknowledgements.
The authors would like to thank Dr. Yuhua Li for providing the MATLAB code of the BEPS method. The authors would also like to thank the anonymous referees for their valuable comments and helpful suggestions. The work is supported by the Sponsor National Natural Science Foundation of China Rlhttp://dx.doi.org/10.13039/501100001809 under Grant No.: Grant #3 and Grant #3.

References

  • (1)
  • Banko et al. (2007) Michele Banko, Michael J. Cafarella, Stephen Soderland, Matt Broadhead, and Oren Etzioni. 2007. Open Information Extraction from the Web. In

    Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI’07)

    . 2670–2676.
  • Berant et al. (2013) Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic Parsing on Freebase from Question-Answer Pairs. In

    Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP’13)

    . 1533–1544.
  • Bollacker et al. (2008) Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: A collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data (SIGMOD’08). 1247–1250.
  • Bordes et al. (2013) Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating Embeddings for Modeling Multi-relational data. In Proceedings of Advances in Neural Information Processing Systems (NIPS’13). 2787–2795.
  • Bordes et al. (2014) Antoine Bordes, Jason Weston, and Sumit Chopra. 2014. Question Answering with Subgraph Embeddings. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 615–620.
  • Brian Murphy and Mitchell (2012) Partha Talukdar Brian Murphy and Tom Mitchell. 2012. Learning effective and interpretable semantic models using non-negative sparse embedding. In Proceedings of COLING 2012. 1933–1950.
  • Chang et al. (2014) Kai-Wei Chang, Wen tau Yih, Bishan Yang, and Christopher Meek. 2014.

    Typed tensor decomposition of knowledge bases for relation extraction. In

    Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP’14). 1568–1579.
  • Dettmers et al. (2017) Tim Dettmers, Minervini Pasquale, Stenetorp Pontus, and Sebastian Riedel. 2017. Convolutional 2D knowledge graph embeddings. In Proceedings of the 2017 AAAI Conference on Artificial Intelligence (AAAI’17). 1811–1818.
  • Ding et al. (2018) Boyang Ding, Quan Wang, Bin Wang, and Li Guo. 2018. Improving Knowledge Graph Embedding Using Simple Constraints. In Proceedings of the 56th Annual Meeting of (ACL’18). Melbourne, Australia, 110–121.
  • Dong et al. (2014) Xin Dong, Evgeniy Gabrilovich, Geremy Heitz, Wilko Horn, Ni Lao, Kevin Murphy, Thomas Strohmann, Shaohua Sun, and Wei Zhang. 2014. Knowledge vault: A web-scale approach to probabilistic knowledge fusion. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (SIGKDD’14). 601–610.
  • Galaarraga et al. (2015) Luis Antonio Galaarraga, Christina Teflioudi, Katja Hose, and Fabian M. Suchanek. 2015. Fast rule mining in ontological knowledge bases with AMIE+. The VLDB Journal 24, 6 (2015), 707–730.
  • Guo et al. (2015) Shu Guo, Quan Wang, Bin Wang, Lihong Wang, and Li Guo. 2015. Semantically smooth knowledge graph embedding. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL’15). 84–94.
  • Guo et al. (2018) Shu Guo, Quan Wang, Lihong Wang, Bin Wang, and Li Guo. 2018. Knowledge graph embedding with iterative guidance from soft rules. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 4816–4823.
  • Heindorf et al. (2016) Stefan Heindorf, Martin Potthast, Benno Stein, and Gregor Engels. 2016. Vandalism detection in wikidata. In International on Conference on Information and Knowledge Management (CIKM’16). ACM, 327–336.
  • Jenatton et al. (2012) Rodolphe Jenatton, Nicolas L. Roux, Antoine Bordes, and Guillaume R. Obozinski. 2012. A latent factor model for highly multi-relational data. In Advances in Neural Information Processing Systems. 3167–3175.
  • Ji et al. (2015) Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Knowledge graph embedding via dynamic mapping matrix. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL’15). 687–696.
  • Jia et al. (2018) Shengbin Jia, Yang Xiang, Xiaojun Chen, and Shijia E. 2018. TTMF: A Triple Trustworthiness Measurement Frame for Knowledge Graphs. (2018).
  • Lai et al. (2015) Siwei Lai, Liheng Xu, Kang Liu, and Jun Zhao. 2015.

    Recurrent Convolutional Neural Networks for Text Classification. In

    Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence. 2267–2273.
  • Lee and Seung (1999) Daniel D. Lee and H. Sebastian Seung. 1999. Learning the parts of objects by non-negative matrix factorization. Nature 401 (1999), 788–791.
  • Lehmann et al. (2015) Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, Dimitris Kontokostas, and Pablo N. Mendes et al. 2015. DBpedia: A largescale, multilingual knowledge base extracted from Wikipedia. Semantic Web Journal 6, 2 (2015), 167–195.
  • Liang et al. (2017) Jiaqing Liang, Yanghua Xiao, Yi Zhang, Seung won Hwang, and Haixun Wang. 2017. Graph-Based Wrong IsA Relation Detection in a Large-Scale Lexical Taxonomy. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI’17). 1178–1184.
  • Lin et al. (2015a) Yankai Lin, Zhiyuan Liu, Huanbo Luan, Maosong Sun, Siwei Rao, and Song Liu. 2015a. Modeling relation paths for representation learning of knowledge bases. In Proceedings of the 2015 EMNLP. 705–714.
  • Lin et al. (2015b) Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015b. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI’15). 2181–2187.
  • Lin et al. (2016) Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. 2016. Neural Relation Extraction with Selective Attention over Instances. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL’16). 2124–2133.
  • Liu et al. (2017) Hanxiao Liu, Yuexin Wu, and Yiming Yang. 2017. Analogical inference for multi-relational embeddings. In Proceedings of the 34th International Conference on Machine Learning (ICML’17). 2168–2178.
  • Melo and Paulheim (2017) Andre Melo and Heiko Paulheim. 2017. Detection of Relation Assertion Errors in Knowledge Graphs. In Proceedings of ACM Conference. 22.
  • Mitra et al. (2002) Sushmita Mitra, Sankar K. Pal, and Pabitra Mitra. 2002. Data Mining in Soft Computing Framework: A Survey. IEEE Transaction on Neural Networks 13, 1 (2002).
  • Nickel et al. (2016a) Maximilian Nickel, Kevin Murphy, Volker Tresp, and Evgeniy Gabrilovich. 2016a. A review of relational machine learning for knowledge graphs. Proc. IEEE 104, 1 (2016), 11–33.
  • Nickel et al. (2016b) Maximilian Nickel, Lorenzo Rosasco, and Tomaso Poggio. 2016b. Holographic embeddings of knowledge graphs. In Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI’16). 1955–1961.
  • Nickel et al. (2011) Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2011. A three-way model for collective learning on multi-relational data. In Proceedings of the 28th International Conference on Machine Learning (ICML’11). 809–816.
  • Nickel et al. (2012) Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2012. Factorizing yago: scalable machine learning for linked data. In Proceedings of the 21st international conference on World Wide Web (WWW’12). ACM, 271–280.
  • Paulheim (2017) Heiko Paulheim. 2017. Knowledge Graph Refinement: A Survey of Approaches and Evaluation Methods. Semantic web 8, 3 (2017), 489–508.
  • Paulheim and Bizer (2014) Heiko Paulheim and Christian Bizer. 2014. Improving the Quality of Linked Data Using Statistical Distributions. International Journal on Semantic Web and Information Systems 10, 2 (2014), 63–86.
  • Shi and Weninger (2017) Baoxu Shi and Tim Weninger. 2017. ProjE: Embedding projection for knowledge graph completion. In Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI’17). 1236–1242.
  • Socher et al. (2013) Richard Socher, Danqi Chen, Christopher D. Manning, and Andrew Y. Ng. 2013. Reasoning with neural tensor networks for knowledge base completion. In Advances in Neural Information Processing Systems (NIPS’13). 926–934.
  • Stanovsky et al. (2007) Gabriel Stanovsky, Julian Michael, Luke Zettlemoyer, and Ido Dagan. 2007. Supervised Open Information Extraction. In Proceedings of NAACL-HLT 2018. 885–895.
  • Tang et al. (2015) Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. Line:Large-scale information network embedding. In Proceedings of the 2015 World Wide Web Conference (WWW’15). 1067–1077.
  • Trouillon et al. (2016) Theo Trouillon, Johannes Welbl, Sebastian Riedel, Eric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction.. In Proceedings of the 33rd International Conference on Machine Learning (ICML’16). 2071–2080.
  • Wang et al. (2017) Quan Wang, Zhendong Mao, Bin Wang, and Li Guo. 2017. Knowledge graph embedding: A survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering 29, 12 (2017), 2724–2743.
  • Wang et al. (2014) Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014.

    Knowledge graph embedding by translating on hyperplanes.. In

    Proceedings of the 28th AAAI Conference on Artificial Intelligence. 1112–1119.
  • Xiao et al. (2016) Han Xiao, Minlie Huang, and Xiaoyan Zhu. 2016. TransG: A generative model for knowledge graph embedding. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL’16). 2316–2325.
  • Xiao et al. (2017) Han Xiao, Minlie Huang, and Xiaoyan Zhu. 2017. SSP: Semantic space projection for knowledge graph embedding with text descriptions. In Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI’17). 3104–3110.
  • Xie et al. (2016b) Ruobing Xie, Zhiyuan Liu, Jia Jia, Huanbo Luan, and Maosong Sun. 2016b. Representation learning of knowledge graphs with entity descriptions. In Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI’16). 2659–2665.
  • Xie et al. (2018) Ruobing Xie, Zhiyuan Liu, Fen Lin, and Leyu Lin. 2018. Does William Shakespeare REALLY Write Hamlet? Knowledge Representation Learning with Confidence. In Proceedding of Association for the Advancement of Artificial Intelligence.
  • Xie et al. (2016a) Ruobing Xie, Zhiyuan Liu, and Maosong Sun. 2016a. Representation learning of knowledge graphs with hierarchical types. In Proceedings of the 25th International Joint Conference on Artificial Intelligence (IJCAI’16). 2965–2971.
  • Zhang et al. (2015) Dongxu Zhang, Bin Yuan, Dong Wang, and Rong Liu. 2015. Joint semantic relevance learning with text data and graph knowledge. (2015), 32–40.
  • Zhao et al. ([n. d.]) Yu Zhao, Sheng Gao, Patrick Gallinari, and Jun Guo. [n. d.]. Knowledge base completion by learning pairwise-interaction differentiated embeddings. Data Mining and Knowledge Discovery 29, 5 ([n. d.]), 1486–1504.
  • Zhong et al. (2015) Huaping Zhong, Jianwen Zhang, Zhen Wang, Hai Wan, and Zheng Chen. 2015. Aligning knowledge and text embeddings by entity descriptions. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP’15). 267–272.