Relation Extraction is a fundamental task in natural language processing (NLP), which aims to extract semantic relations between entities. For example, sentence “[Barack Obama] was born in [Hawaii]” expresses the relation BornIn between entity pair Barack Obama and Hawaii.
|B1||S1. Barack Obama was born in the United States.||Yes|
|S2. Barack Obama was the 44th president of the United States||No|
|B2||S3. Kyle Busch , a Las Vegas resident who ran second to Johnson last year, finished third, followed by Kasey Kahne, Jeff Gordon and mark martin .||No|
|S4. Hendrick drivers finished in three of the top four spots at Las Vegas , including Kyle Busch in second and …||No|
Conventional relation extraction methods, such as (Zelenko et al., 2002; Culotta and Sorensen, 2004; Mooney and Bunescu, 2006), adopted supervised training and suffered from the lack of large-scale manually labeled data. To address this issue, the distant supervision method (Mintz et al., 2009) was proposed, which generated the data for training relation extraction models automatically. The distant supervision assumption says that if two entities participate in a relation, all sentences that mention these two entities express that relation. It is inevitable that there exists noise in the data labeled by distant supervision. For example, the precision of aligning the relations in Freebase to the New York Times corpus was only about 70% (Riedel et al., 2010).
Thus, the relation extraction method proposed in (Riedel et al., 2010) argued that the distant supervision assumption was too strong and relaxed it to expressed-at-least-once assumption. This assumption says that if two entities participate in a relation, at least one sentence that mentions these two entities might express that relation. An example is shown by sentences S1 and S2 in Table 1. This relation extraction method first divided the training data given by distant supervision into bags where each bag was a set of sentences containing the same entity pair. Then, bag representations were derived by weighting sentences within each bag. It was expected that the weights of the sentences with incorrect labels were reduced and the bag representations were calculated mainly using the sentences with correct labels. Finally, bags were utilized as the samples for training relation extraction models instead of sentences.
In recent years, many relation extraction methods using neural networks with attention mechanism (Lin et al., 2016; Ji et al., 2017; Jat et al., 2018) have been proposed to alleviate the influence of noisy training data under the expressed-at-least-once assumption. However, these methods still have two deficiencies. First, only the target relation of each bag is used to calculate the attention weights for deriving bag representations from sentence embeddings at training stage. Here we argue that the bag representations should be calculated in a relation-aware way. For example, the bag B1 in Table 1
contains two sentences S1 and S2. When this bag is classified to relationBornIn, the sentence S1 should have higher weight than S2, but when classified to relation PresidentOf, the weight of S2 should be higher. Second, the expressed-at-least-once assumption ignores the noisy bag problem which means that all sentences in one bag are incorrectly labeled. An example is shown by bag B2 in Table 1.
In order to deal with these two deficiencies of previous methods, this paper proposes a neural network with multi-level attentions for distant supervision relation extraction. At the instance/sentence-level, i.e., intra-bag level, all possible relations are employed as queries to calculate the relation-aware bag representations instead of only using the target relation of each bag. To address the noisy bag problem, a bag group is adopted as a training sample instead of a single bag. Here, a bag group is composed of bags in the training set which share the same relation label. The representation of a bag group is calculated by weighting bag representations using a similarity-based inter-bag attention module.
The contributions of this paper are threefold. First, an improved intra-bag attention mechanism is proposed to derive relation-aware bag representations for relation extraction. Second, an inter-bag attention module is introduced to deal with the noisy bag problem which is ignored by the expressed-at-least-once assumption. Third, our methods achieve better extraction accuracy than state-of-the-art models on the widely used New York Times (NYT) dataset (Riedel et al., 2010).
2 Related Work
treated relation extraction as a supervised learning task and designed hand-crafted features to train kernel-based models. Due to the lack of large-scale manually labeled data for supervised training, the distant supervision approach(Mintz et al., 2009) was proposed, which aligned raw texts toward knowledge bases automatically to generate relation labels for entity pairs. However, this approach suffered from the issue of noisy labels. Therefore, some subsequent studies (Riedel et al., 2010; Hoffmann et al., 2011; Surdeanu et al., 2012) considered distant supervision relation extraction as a multi-instance learning problem, which extracted relation from a bag of sentences instead of a single sentence.
With the development of deep learning techniques(LeCun et al., 2015), many neural-network-based models have been developed for distant supervision relation extraction. Zeng et al. (2015)
proposed piecewise convolutional neural networks (PCNNs) to model sentence representations and chose the most reliable sentence as the bag representation.Lin et al. (2016) employed PCNNs as sentence encoders and proposed an intra-bag attention mechanism to compute the bag representation via a weighted sum of all sentence representations in the bag. Ji et al. (2017) adopted a similar attention strategy and combined entity descriptions to calculate the weights. Liu et al. (2017)
proposed a soft-label method to reduce the influence of noisy instances. All these methods represented a bag with a weighted sum of sentence embeddings, and calculated the probability of the bag being classified into each relation using the same bag representation at training stage. In our proposed method, intra-bag attentions are computed in a relation-aware way, which means that different bag representations are utilized to calculate the probabilities for different relation types. Besides, these existing methods focused on intra-bag attentions and ignored the noisy bag problem.
both employed reinforcement learning to train instance selector and to filter out the samples with wrong labels. Their rewards were calculated from the prediction probabilities and the performance change of the relation classifier respectively.Qin et al. (2018a) designed an adversarial learning process to build a sentence-level generator via policy-gradient-based reinforcement learning. These methods were proposed to filter out the noisy data at sentence-level and also failed to deal with the noisy bag problem explicitly.
In this section, we introduce a neural network with intra-bag and inter-bag attentions for distant supervision relation extraction. Let denote a group of bags which have the same relation label given by distant supervision, and is the number of bags within this group. Let denote all sentences in bag , and is the number of sentences in bag . Let denote the -th sentence in the -th bag and is its length (i.e., number of words). The framework of our model is shown in Fig. 1, which has three main modules.
Sentence Encoder Given a sentence and the positions of two entities within this sentence, CNNs or PCNNs (Zeng et al., 2015) are adopted to derive the sentence representation .
Intra-Bag Attention Given the sentence representations of all sentences within a bag and a relation embedding matrix
, attention weight vectorsand bag representations are calculated for all relations, where is the relation index.
Inter-Bag Attention Given the representations of all bags with the group , a weight matrix is further calculated via similarity-based attention mechanism to obtain the representation of the bag group.
More details of these three modules will be introduced in the following subsections.
3.1 Sentence Encoder
3.1.1 Word Representation
Each word within the sentence is first mapped into a -dimensional word embedding. To describe the position information of two entities, the position features (PFs) proposed in (Zeng et al., 2014) are also adopted in our work. For each word, the PFs describe the relative distances between current word and the two entities and are further mapped into two vectors and of dimensions. Finally, these three vectors are concatenated to get the word representation of dimensions.
3.1.2 Piecewise CNN
For sentence , the matrix of word representations is first input into a CNN with
filters. Then, piecewise max pooling(Zeng et al., 2015) is employed to extract features from the three segments of CNN outputs, and the segment boundaries are determined by the positions of the two entities. Finally, the sentence representation can be obtained.
3.2 Intra-Bag Attention
Let represent the representations of all sentences within bag , and denote a relation embedding matrix where is the number of relations.
Different from conventional methods (Lin et al., 2016; Ji et al., 2017) where a unified bag representation was derived for relation classification, our method calculates bag representations for bag on the condition of all possible relations as
where is the relation index and is the attention weight between the -th relation and the -th sentence in bag . can be further defined as
where is the matching degree between the -th relation query and the -th sentence in bag . In our implementation, a simple dot product between vectors is adopted to calculate the matching degree as
where is the -th row of the relation embedding matrix 222We also tried , where was a diagonal matrix, in experiments and achieved similar performance..
Finally, the representations of bag compose the matrix in Fig. 1, where each row corresponds to a possible relation type of this bag.
3.3 Inter-Bag Attention
In order to deal with the noisy bag problem, a similarity-based inter-bag attention module is designed to reduce the weights of noisy bags dynamically. Intuitively, if two bags and are both labeled as relation , their representations and should be close to each other. Given a group of bags with the same relation label, we assign higher weights to those bags which are close to other bags in this group. As a result, the representation of bag group can be formulated as
where is the -th row of the matrix in Fig. 1, is the relation index and composes the attention weight matrix . Each is defined as
where describes the confidence of labeling bag with the -th relation.
Inspired by the self-attention algorithm (Vaswani et al., 2017) which calculates the attention weights for a group of vectors using the vectors themselves, we calculate the weights of bags according to their own representations. Mathematically, is defined as
where the function similarity is a simple dot product in our implementation as
And also, in order to prevent the influence of vector length, all bag representations are normalized to unit length as before calculating Eq.(4)-(7).
Then, the score of classifying bag group into relation is calculated via and relation embedding as
where is a bias term. Finally, a softmax function is employed to obtain the probability that the bag group is classified into the -th relation as
It should be noticed that the same relation embedding matrix is used for calculating Eq.(3) and Eq.(8). Similar to Lin et al. (2016), the dropout strategy (Srivastava et al., 2014) is applied to bag representation to prevent overfitting.
3.4 Implementation Details
3.4.1 Data Packing
First of all, all sentences in the training set that contain the same two entities are accumulated into one bag. Then, we tie up every bags that share the same relation label into a group. It should be noticed that a bag group is one training sample in our method. Therefore, the model can also be trained in mini-batch mode by packing multiple bag groups into one batch.
3.4.2 Objective Function and Optimization
In our implementation, the objective function is defined as
where is the set of all training samples and
is the set of model parameters, including word embedding matrix, position feature embedding matrix, CNN weight matrix and relation embedding matrix. The model parameters are estimated by minimizing the objective function J(
) through mini-batch stochastic gradient descent (SGD).
3.4.3 Training and Test
As introduced above, at the training phase of our proposed method, bags which have the same relation label are accumulated into one bag group and the weighted sum of bag representations is calculated to obtain the representation of the bag group. Due to the fact that the label of each bag is unknown at test stage, each single bag is treated as a bag group (i.e., =1) when processing the test set.
And also, similar to (Qin et al., 2018b), we only apply inter-bag attentions to positive samples, i.e., the bags whose relation label is not (). The reason is that the representations of the bags that express no relations are always diverse and it’s difficult to calculate suitable weights for them.
3.4.4 Pre-training Strategy
In our implementation, a pre-training strategy is adopted. We first train the model with only intra-bag attentions until convergence. Then, the inter-bag attention module is added and the model parameters are further updated until convergence again. Preliminary experimental results showed that this strategy can lead to better model performance than considering inter-bag attentions from the very beginning.
|position feature||max relative distance||30|
4.1 Dataset and Evaluation Metrics
The New York Times (NYT) dataset was adopted in our experiments. This dataset was first released by Riedel et al. (2010) and has been widely used by previous research on distant supervision relation extraction Liu et al. (2017); Jat et al. (2018); Qin et al. (2018a, b). This dataset was generated by aligning Freebase with the New York Times (NYT) corpus automatically. There were 52 actual relations and a special relation which indicated there was no relation between two entities.
Following previous studies (Mintz et al., 2009; Liu et al., 2017), we evaluated our models on the held-out test set of the NYT dataset. Precision-recall (PR) curves, area under curve (AUC) values and Precision@N (P@N) values (Lin et al., 2016)
were adopted as evaluation metrics in our experiments. All of the numerical results given by our experiments were the mean values of 10 repetitive trainings, and the PR curves were randomly selected from the repetitions because there was no significant visual difference among them.
4.2 Training Details and Hyperparameters
All of the hyperparameters used in our experiments are listed in Table2. Most of them followed the hyperparameter settings in (Lin et al., 2016). The 50-dimensional word embeddings released by (Lin et al., 2016)333https://github.com/thunlp/NRE. were also adopted for initialization. The vocabulary contained the words which appeared more than 100 times in the NYT corpus.
Two different batch sizes and were used for pre-training and training respectively. In our experiments, a grid search is employed using training set to determine the optimal values of , and among , and . Note that increasing the bag group size may boost the effect of inter-bag attentions but lead to less training samples. The effects of inter-bag attentions would be lost when =1. For optimization, we employed mini-batch SGD with the initial learning rate of 0.1. The learning rate was decayed to one tenth every 100,000 steps. The pre-trained model with only intra-bag attentions converged within 300,000 steps in our experiments. Thus, the initial learning rate for training the model with inter-bag attentions was set as 0.001.
4.3 Overall performance
Eight models were implemented for comparison. The names of these models are listed in Table 3, where CNN and PCNN denote using CNNs or piecewise CNNs in sentence encoders respectively, ATT_BL means the baseline intra-bag attention method proposed by (Lin et al., 2016), ATT_RA means our proposed relation-aware intra-bag attention method, and BAG_ATT means our proposed inter-bag attention method. At the training stage of the ATT_BL
method, the relation query vector for attention weight calculation was fixed as the embedding vector associated with the distant supervision label for each bag. At the test stage, all relation query vectors were applied to calculate the posterior probabilities of relations respectively and the relation with the highest probability was chosen as the classification result(Lin et al., 2016)
. The means and standard deviations of the AUC values given by the whole PR curves of these models are shown in Table3 for a quantitative comparison. Following (Lin et al., 2016), we also plotted the PR curves of these models in Fig. 2 and 3 with recall smaller than for a visualized comparison.
|# of Test Sentences||one||two||all|
|Lin et al. (2016)||73.3||69.2||60.8||67.8||77.2||71.6||66.1||71.6||76.2||73.1||67.4||72.2|
|Liu et al. (2017)||84.0||75.5||68.3||75.9||86.0||77.0||73.3||78.8||87.0||84.5||77.0||82.8|
From Table 3, Fig. 2 and Fig. 3, we have the following observations. (1) Similar to the results of previous work Zeng et al. (2015), PCNNs worked better than CNNs as sentence encoders. (2) When using either CNN or PCNN sentence encoders, ATT_RA outperformed ATT_BL. It can be attributed to that the ATT_BL method only considered the target relation when deriving bag representations at training time, while the ATT_RA method calculated intra-bag attention weights using all relation embeddings as queries, which improved the flexibility of bag representations. (3) For both sentence encoders and both intra-bag attention methods, the models with BAG_ATT always achieved better performances than the ones without BAG_ATT. This result verified the effectiveness of our proposed inter-bag attention method for distant supervision relation extraction. (4) The best AUC performance was achieved by combining PCNN sentence encoders with the intra-bag and inter-bag attentions proposed in this paper.
4.4 Comparison with previous work
4.4.1 PR curves
The PR curves of several models in previous work and our best model PCNN+ATT_RA+BAG_ATT are compared in Fig. 4, where Mintz Mintz et al. (2009), MultiR Hoffmann et al. (2011) and MIMLRE Surdeanu et al. (2012) are conventional feature-based methods, and Lin et al. (2016) and Liu et al. (2017) are PCNN-based ones444All of these curve data are from https://github.com/tyliupku/soft-label-RE.. For a fair comparison with Lin et al. (2016) and Liu et al. (2017), we also plotted the curves with only the top 2000 points. We can see that our model achieved better PR performance than all the other models.
4.4.2 AUC values
ATT_BL+DSGAN Qin et al. (2018a) and ATT_BL+RL Qin et al. (2018b) are two recent studies on distant supervision relation extraction with reinforcement learning for data filtering, which reported the AUC values of PR curves composed by the top 2000 points. Table 5 compares the AUC values reported in these two papers and the results of our proposed models. We can see that introducing the proposed ATT_RA and BAG_ATT methods to baseline models achieved larger improvement than using the methods proposed in Qin et al. (2018a, b).
|bag||sentence||correct?||intra-bag weights||inter-bag weights|
|B1||[Panama City Beach] , too , has a glut of condos , but the area was one of only two in [Florida] where sales rose in march , compared with a year earlier.||Yes||0.71||0.48|
|Like much of [Florida] , [Panama City Beach] has been hurt by the downturn in the real estate market.||Yes||0.29|
|B2||Among the major rivers that overflowed were the Housatonic , Still , Saugatuck , Norwalk , Quinnipiac , Farmington , [Naugatuck] , Mill , Rooster and [Connecticut] .||No||1.00||0.13|
|B3||…, the army chose a prominent location in [Virginia], at the foot of the Arlington memorial bridge , directly across the [Potomac River] from the Lincoln memorial .||No||0.13||0.39|
|… , none of those stars carried the giants the way barber did at Fedex field , across the [Potomac River] from [Virginia] , where he grew up as a redskins fan .||Yes||0.87|
4.5 Effects of Intra-Bag Attentions
Following Lin et al. (2016), we evaluated our models on the entity pairs with more than one training sentence. One, two and all sentences for each test entity pair were randomly selected to construct three new test sets. The P@100, P@200, P@300 values and their means given by our proposed models on these three test sets are reported in Table 4 together with the best results of Lin et al. (2016) and Liu et al. (2017). Here, P@N means the precision of the relation classification results with the top highest probabilities in the test set.
We can see our proposed methods achieved higher P@N values than previous work. Furthermore, no matter whether PCNN or BAG_ATT were adopted, the ATT_RA method outperformed the ATT_BL method on the test set with only one sentence for each entity pair. Note that the decoding procedures of ATT_BL and ATT_RA were equivalent when there was only one sentence in a bag. Therefore, the improvements from ATT_BL to ATT_RA can be attributed to that ATT_RA calculated intra-bag attention weights in a relation-aware way at the training stage.
4.6 Distributions of Inter-Bag Attention Weights
We divided the training set into 5 parts according to the number of sentences in each bag. For each bag, the inter-bag attention weights given by the PCNN+ATT_RA+BAG_ATT model were recorded. Then, the mean and standard deviation of inter-bag attention weights for each part of the training set were calculated and are shown in Table 7. From this table, we can see that the bag with smaller number of training sentences were usually assigned with lower inter-bag attention weights. This result was consistent with the finding in Qin et al. (2018b) that the entity pairs with fewer training sentences were more likely to have incorrect relation labels.
4.7 Case Study
A test set example of relation /location/location/contains is shown in Table 6. The bag group contained 3 bags, which consisted of 2, 1, and 2 sentences respectively. We calculated the intra-bag and inter-bag attentions for this bag group using our PCNN+ATT_RA+BAG_ATT model and the weights of the target relation are also shown in Table 6.
In this example, the second bag was a noisy bag because the only sentence in this bag didn’t express the relation /location/location/contains between the two entities Naugatuck and Connecticut. In conventional methods, these three bags were treated equally for model training. After introducing inter-bag attention mechanism, the weight of this noisy bag was reduced significantly as shown in the last column of Table 6.
In this paper, we have proposed a neural network with intra-bag and inter-bag attentions to cope with the noisy sentence and noisy bag problems in distant supervision relation extraction. First, relation-aware bag representations are calculated by a weighted sum of sentence embeddings where the noisy sentences are expected to have smaller weights. Further, an inter-bag attention module is designed to deal with the noisy bag problem by calculating the bag-level attention weights dynamically during model training. Experimental results on New York Times dataset show that our models achieved significant and consistent improvements compared with the models using only conventional intra-bag attentions. To deal with the multi-label problem of relation extraction and to integrate external knowledge into our model will be the tasks of our future work.
We thank the anonymous reviewers for their valuable comments.
- Culotta and Sorensen (2004) Aron Culotta and Jeffrey Sorensen. 2004. Dependency tree kernels for relation extraction. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04).
- Feng et al. (2018) Jun Feng, Minlie Huang, Li Zhao, Yang Yang, and Xiaoyan Zhu. 2018. Reinforcement learning for relation classification from noisy data. In Proceedings of AAAI.
- Hoffmann et al. (2011) Raphael Hoffmann, Congle Zhang, Xiao Ling, Luke Zettlemoyer, and Daniel S. Weld. 2011. Knowledge-based weak supervision for information extraction of overlapping relations. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 541–550. Association for Computational Linguistics.
- Jat et al. (2018) Sharmistha Jat, Siddhesh Khandelwal, and Partha Talukdar. 2018. Improving distantly supervised relation extraction using word and entity based attention. arXiv preprint arXiv:1804.06987.
- Ji et al. (2017) Guoliang Ji, Kang Liu, Shizhu He, Jun Zhao, et al. 2017. Distant supervision for relation extraction with sentence-level attention and entity descriptions. In AAAI, pages 3060–3066.
- LeCun et al. (2015) Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature, 521(7553):436.
- Lin et al. (2016) Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. 2016. Neural relation extraction with selective attention over instances. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2124–2133. Association for Computational Linguistics.
- Liu et al. (2017) Tianyu Liu, Kexiang Wang, Baobao Chang, and Zhifang Sui. 2017. A soft-label method for noise-tolerant distantly supervised relation extraction. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1790–1795. Association for Computational Linguistics.
- Mintz et al. (2009) Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2-Volume 2, pages 1003–1011. Association for Computational Linguistics.
- Mooney and Bunescu (2006) Raymond J Mooney and Razvan C Bunescu. 2006. Subsequence kernels for relation extraction. In Advances in neural information processing systems, pages 171–178.
- Qin et al. (2018a) Pengda Qin, Weiran XU, and William Yang Wang. 2018a. Dsgan: Generative adversarial training for distant supervision relation extraction. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 496–505. Association for Computational Linguistics.
- Qin et al. (2018b) Pengda Qin, Weiran XU, and William Yang Wang. 2018b. Robust distant supervision relation extraction via deep reinforcement learning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2137–2147. Association for Computational Linguistics.
Riedel et al. (2010)
Sebastian Riedel, Limin Yao, and Andrew McCallum. 2010.
Modeling relations and their mentions without labeled text.
Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 148–163. Springer.
- Srivastava et al. (2014) Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1):1929–1958.
- Surdeanu et al. (2012) Mihai Surdeanu, Julie Tibshirani, Ramesh Nallapati, and Christopher D. Manning. 2012. Multi-instance multi-label learning for relation extraction. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 455–465. Association for Computational Linguistics.
- Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, pages 5998–6008.
- Zelenko et al. (2002) Dmitry Zelenko, Chinatsu Aone, and Anthony Richardella. 2002. Kernel methods for relation extraction. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002).
- Zeng et al. (2015) Daojian Zeng, Kang Liu, Yubo Chen, and Jun Zhao. 2015. Distant supervision for relation extraction via piecewise convolutional neural networks. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1753–1762. Association for Computational Linguistics.
- Zeng et al. (2014) Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, and Jun Zhao. 2014. Relation classification via convolutional deep neural network. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 2335–2344. Dublin City University and Association for Computational Linguistics.