DeepAI
Log In Sign Up

A Multi-turn Machine Reading Comprehension Framework with Rethink Mechanism for Emotion-Cause Pair Extraction

Emotion-cause pair extraction (ECPE) is an emerging task in emotion cause analysis, which extracts potential emotion-cause pairs from an emotional document. Most recent studies use end-to-end methods to tackle the ECPE task. However, these methods either suffer from a label sparsity problem or fail to model complicated relations between emotions and causes. Furthermore, they all do not consider explicit semantic information of clauses. To this end, we transform the ECPE task into a document-level machine reading comprehension (MRC) task and propose a Multi-turn MRC framework with Rethink mechanism (MM-R). Our framework can model complicated relations between emotions and causes while avoiding generating the pairing matrix (the leading cause of the label sparsity problem). Besides, the multi-turn structure can fuse explicit semantic information flow between emotions and causes. Extensive experiments on the benchmark emotion cause corpus demonstrate the effectiveness of our proposed framework, which outperforms existing state-of-the-art methods.

READ FULL TEXT VIEW PDF

page 1

page 2

page 3

page 4

06/04/2019

Emotion-Cause Pair Extraction: A New Task to Emotion Analysis in Texts

Emotion cause extraction (ECE), the task aimed at extracting the potenti...
03/02/2021

An End-to-End Network for Emotion-Cause Pair Extraction

The task of Emotion-Cause Pair Extraction (ECPE) aims to extract all pot...
05/04/2022

Multi-Granularity Semantic Aware Graph Model for Reducing Position Bias in Emotion-Cause Pair Extraction

The Emotion-Cause Pair Extraction (ECPE) task aims to extract emotions a...
09/09/2022

Joint Alignment of Multi-Task Feature and Label Spaces for Emotion Cause Pair Extraction

Emotion cause pair extraction (ECPE), as one of the derived subtasks of ...
04/15/2021

A Dual-Questioning Attention Network for Emotion-Cause Pair Extraction with Context Awareness

Emotion-cause pair extraction (ECPE), an emerging task in sentiment anal...
08/18/2017

A Question Answering Approach to Emotion Cause Extraction

Emotion cause extraction aims to identify the reasons behind a certain e...
06/04/2019

RTHN: A RNN-Transformer Hierarchical Network for Emotion Cause Extraction

The emotion cause extraction (ECE) task aims at discovering the potentia...

1 Introduction

(a) An example of the ECPE task
(b) Pairing matrix
(c) Our approach
Figure 1: The green colour denotes an emotion clause, and the red colour denotes a cause clause. Figure (a) is an example of the ECPE task. Figure (b) is a pairing matrix generated by pair-level end-to-end approaches. Only (c2, c3) and (c4, c4) are valid pairs. Figure (c) shows the processing results by our MM-R in each turn.

Emotion cause extraction (ECE) is a classical emotion cause analysis task that aims to extract the corresponding causes of the given emotional expressions in an emotional document  Gui et al. (2016). However, the ECE task is not practical in real-world scenarios without annotated emotions. To overcome the limitation and capture mutual indications of emotions and causes together, Xia and Ding (2019) came up with a new task called emotion-cause pair extraction (ECPE), which aims to extract a potential emotion-cause pair set consisting of all emotions and their corresponding causes from a document. As shown in Figure 1(a), c2 is an emotion clause, c3 is the corresponding cause clause, and c4 is an emotion clause that is its own corresponding cause clause itself.

In recent years, there has been a trend to use end-to-end methods to solve the ECPE task and the two sub-tasks, emotion extraction and cause extraction, which aim to extract all emotions and causes in one document. These end-to-end methods can be classified into two categories: pair-level methods by combining all clause pairs to form a clause pairing matrix; sequence labeling methods by designing novel tagging schemes. Specifically, some works  

Ding et al. (2020a, b); Wei et al. (2020); Chen et al. (2020b); Wu et al. (2020) proposed pair-level methods that generate a pairing matrix by enumerating all possible combinations of clauses and then select valid emotion-cause pairs, as shown in Figure 1(b). However, the average ratio of invalid/valid pairs is more than 200:1 in the ECPE benchmark corpus Xia and Ding (2019), which leads to a label sparsity problem. Besides, some works Chen et al. (2020a); Yuan et al. (2020) proposed sequence labeling methods based on novel tagging schemes. However, they cannot effectively model the corresponding relation between emotions and causes. For example, Chen et al. (2020a) cannot deal with the situation where two emotion clauses of the same emotion type correspond to different cause clauses. Therefore, how to solve the label sparsity problem by avoiding the calculation of the pairing matrix while modeling the corresponding relation between emotions and causes is a crucial challenge. Furthermore, Zhong and Chen (2021) has shown that explicit semantic information can improve performance in the extraction tasks. Therefore, utilizing explicit semantic information of clauses to improve the performance in the ECPE task is also a challenge.

To address these challenges, in this study, the ECPE task is transformed into a document-level machine reading comprehension (MRC) task, and a Multi-turn MRC framework with Rethink mechanism (MM-R) is proposed. The multi-turn structure decomposes the ECPE task to model the corresponding relation between emotions and causes and avoid generating the pairing matrix. In every turn, static and dynamic queries designed manually make full use of explicit semantic information of clauses to improve the performance of emotion or cause extraction. In addition, inspired by the human two-stage reading behavior, in which search for possible answer candidates and then verify these candidates Zheng et al. (2019), the rethink mechanism is proposed to verify candidate emotion-cause pairs further to enhance the flow of information between emotions and causes and improve the overall performance.

Specifically, in the first turn, all emotion clauses are extracted. In the second turn, cause clauses corresponding to each emotion clause are extracted with the assistance of explicit semantic information about emotions. Numerous experiments Ding et al. (2020b); Fan et al. (2020a); Ding et al. (2020a); Wei et al. (2020); Chen et al. (2020b, a); Yuan et al. (2020) have shown that emotion extraction is more reliable than cause extraction, and therefore it is reasonable to extract emotions first and then extract the corresponding causes with the assistance of emotions. Hence, a candidate emotion-cause pair set is obtained in the first two turns. In the third turn, a rethink mechanism verifies each candidate emotion-cause pair. For example, as shown in Figure 1(c), emotion clauses c2 and c4 can be obtained in the first turn, and cause clauses corresponding to each emotion clause can be obtained in the second turn. The candidate emotion-cause pair set {(c2, c3), (c2, c4), (c4, c4)} can thus be obtained without the pairing matrix. In the third turn, the rethink mechanism demonstrates that c2 is not one of the corresponding emotion clauses of c4. Therefore, the valid emotion-cause pair set becomes {(c2, c3), (c4, c4)}.

Comprehensive experiments are conducted on the ECPE benchmark datasets to verify the effectiveness of the proposed MM-R framework. The experimental results show that the proposed framework substantially outperforms the previous methods. The contributions of this research can be summarized as follows:

  • The ECPE task is formalized as a document-level machine reading comprehension (MRC) task. To our best knowledge, this is the first time that the ECPE task has been transferred to the MRC task.

  • Based on the MRC formalization, a multi-turn MRC framework with rethink mechanism (MM-R) is proposed. It models the corresponding relation between emotions and causes and alleviates the label sparsity problem. Furthermore, the MM-R can use explicit semantic information of clauses effectively.

  • The experimental results demonstrate that the proposed framework outperforms existing state-of-the-art methods.

2 Related Work

2.1 Emotion-Cause Pair Extraction

Xia and Ding (2019) proposed the emotion-cause pair extraction (ECPE) task and released the Chinese benchmark dataset. Recently, many methods based on the pair-level end-to-end framework have been designed for the ECPE task. For example, Wei et al. (2020) obtained excellent results by modeling inter-clause relationships and using the ranking mechanism. Ding et al. (2020a) used a 2D-transformers module to model the interactions of different emotion-cause pairs. However, these pair-level end-to-end methods had a label sparsity problem caused by calculating the pairing matrix. Ding et al. (2020b) used a sliding window to restrict the amount of candidate emotion-cause pairs, which aims to shrink the pairing matrix. However, the method still belonged to the pair-level end-to-end method because it assumed all clauses were emotion or cause clauses.

Sequence labeling was another popular method. However, these methods could not model the corresponding relation between emotions and causes. A unified tagging scheme Chen et al. (2020a) could not be used on a document with multiple emotion clauses of the same emotion types because the tagging scheme was based on emotion types. Another joint tagging scheme Yuan et al. (2020) based on the distance between the cause and the corresponding triggered emotion. The tagging scheme could not handle the situation in which the distance between the emotion clause and the corresponding cause clause exceeded the distance threshold. Therefore, the challenge remains of how to solve the label sparsity problem while modeling the corresponding relation between emotions and causes.

2.2 Machine Reading Comprehension

Recently, it became popular to transform many traditional natural language processing tasks into machine reading comprehension (MRC) tasks. For example,

Liu et al. (2020) cast the event extraction as a machine reading comprehension problem to solve the data scarcity problem. Zhou et al. (2021) proposed a dual QA framework aiming at event argument extraction. Chen et al. (2021) and Mao et al. (2021)

proposed a bidirectional MRC and a Dual-MRC frameworks to handle aspect-based sentiment analysis, respectively. In this paper, the ECPE task is formalized as a document-level MRC task, and the multi-turn MRC framework with rethink mechanism is proposed. It concatenates a query with each clause to explicitly utilize semantic information of clauses and uses a single classifier to predict whether a clause is an emotion or a cause clause in each turn.

3 Methodology

3.1 Problem Formulation

Given a document consisting of multiple clauses 222The denotes the number of elements in the collection . and each clause contains multiple words , our goal is to extract a set of emotion-cause pair in :

(1)

where the superscript and denote the -th emotion clause and its corresponding -th cause clause in ; the subscript denotes the -th emotion-cause pair in set ; an emotion clause corresponds to one or more cause clauses. 333To avoid confusion, the “cause clause” is denoted to rather than .

3.2 Query Design

Static and dynamic queries are used to formalize the ECPE task to the MRC task. All queries can be formulated as follows:444Static cause query and static pair query are only used in two variants of our framework (MM-D and MRC-E2E); refer to the Experiments section for details.

  • Static emotion query : The query “Is it an emotion clause?” is designed to extract all emotion clauses.

  • Static cause query : The query “Is it a cause clause?” is designed to extract all cause clauses.

  • Static pair query : The query “Is it an emotion-cause pair?” is designed to extract all emotion-cause pairs.

  • Dynamic emotion query : The query template “Is it an emotion clause corresponding to ?” is designed to extract emotion clauses corresponding to clause .

  • Dynamic cause query : The query template “Is it a cause clause corresponding to ?” is designed to extract cause clauses corresponding to clause .

Figure 2: Left

: The overall architecture of our MM-R framework. In each turn, the answer is yes if the probability output by the classifier is greater than 0.5, otherwise it is no.

Right

: The implementation structure of the encoding layer which includes the token-level encoder and the clause-level encoder. The token-level encoder generates the hidden representation of each token using the BERT module. The clause-level encoder provides the hidden representation of query and each clause using the attention mechanism and graph attention network. Finally, the concatenate operation (CONCAT) is executed on the hidden representations of queries and clauses.

3.3 Frameworks

The architecture of MM-R is illustrated on the left side of Figure 2. In the first turn, a static emotion query is used to extract emotion clause set . In the second turn, based on each extracted emotion clause , a dynamic cause query is constructed to obtain ’s corresponding cause clause set . In this way, the candidate emotion-cause pair set is obtained in the first two turns. To filter out invalid emotion-cause pairs in further, for each cause clause , the framework rethinks whether the clause is its corresponding emotion clause with the help of dynamic emotion query . Finally, the valid emotion-cause pair set is obtained.

3.4 Encoding Layer

The structure of the encoding layer is illustrated on the right side of Figure 2. The encoding layer includes the token-level encoding layer and the clause-level encoding layer.

Token-level encoder: This study used BERT Devlin et al. (2019) as its token-level contextualized encoder. Specifically, a document and a query are used to construct the BERT input sequence:

(2)

where in the first turn, in the second turn and in the third turn; is the -th token of query ; is the -th token of the -th clause in the document ; and are special BERT tokens. The tokens can then be encoded into hidden representations:

(3)

where ; is the dimension of the hidden states; and denotes the hidden representation of token .

Clause-level encoder: The attention mechanism and the graph attention network (GAT) Veličković et al. (2017) are used to obtain clause-level hidden representations.

Specifically, for clause , its token-level representation set is selected from :

(4)

The attention mechanism produces an attention weight vector

. The hidden representation of clause is obtained by calculating the weighted sum of the hidden representations of all tokens:

(5)
(6)

where and are learnable parameters. For all clauses, the attention mechanism is used to obtain the hidden representations:

(7)

Similarly, the attention mechanism is used to obtain the hidden representation of query :

(8)

Furthermore, GAT models the interaction among clauses, and then the hidden representations of clauses are updated to:

(9)

Finally, and are concatenated to obtain , where denotes the concatenate operation. The output of the encoding layer is thus obtained:

(10)

3.5 Answering Prediction

The output representation of the encoder layer contains abundant semantic information, including identification information of emotion/cause, semantic information of clauses, and semantic information of the document. Therefore, only a single linear perception (SLP) is needed to obtain the answer to the query by predicting yes or no:

(11)

where and are learnable parameters of SLP; is a logistic function; and denotes the probability that the answer is yes. If , the answer is judged to be yes, meaning that clause is one of the answers to query . This prediction style is similar to BoolQ Clark et al. (2019).

3.6 Joint Training

A joint learning strategy was used in this study, the cross-entropy loss for each type of query was minimized as follows:

(12)

where ; denotes the number of documents in the dataset; is the -th clause of the -th document; and is the -th query in .

Therefore, the final loss function

was as follows:

(13)

3.7 Inference

In the first turn, the first identifies the emotion clause set . In the second turn, for each predicted emotion clause , the recognizes the corresponding cause clause set and obtains the set of candidate emotion-cause pairs . And the probability of each candidate pair is .

In the third turn, the is used for selecting valid emotion-cause pairs. Furthermore, the rethink mechanism is implemented through a soft selection strategy, in which the probability is adjusted and a probability threshold is used. Specifically, the probability of the candidate pair is updated by this strategy as follows:

(14)

where the weight factor is used to adjust the probability of candidate emotion-cause pairs. is 1 when the predicted result of the third turn is yes, otherwise is a unique value between 0 and 1. The set of valid emotion-cause pairs is as follows:

(15)

where is a probability threshold value.

Model E-C Pair Extraction Emotion Extraction Cause Extraction
P(%) R(%) F1(%) P(%) R(%) F1(%) P(%) R(%) F1(%)
SL-NTS 72.43 63.66 67.76 81.96 73.29 77.39 74.90 66.02 70.18
TransDGC (Val) 73.74 63.07 67.99 87.16 82.44 84.74 75.62 64.71 69.74
ECPE-2D 72.92 65.44 68.89 86.27 92.21 89.10 73.36 69.34 71.23
PairGCN 76.92 67.91 72.02 88.57 79.58 83.75 79.07 69.28 73.75
RANKCP 71.19 76.30 73.60 91.23 89.99 90.57 74.61 77.88 76.15
ECPE-MLL 77.00 72.35 74.52 86.08 91.91 88.86 73.82 79.12 76.30
MM-R 82.18 79.27 80.62 97.38 90.38 93.70 83.28 79.64 81.35
MM-R (Val) 78.97 75.32 77.06 96.09 88.09 91.88 80.90 76.21 78.45
Table 1: Performance of our models and baselines. P, R and F1 denote precision, recall and F1-measure respectively. E-C denotes Emotion-Cause. TransDGC(Val) and MM-R(Val) use the second data split style, the rest of models use the first data split style.

4 Experiments

4.1 Dataset and Metrics

The benchmark dataset Xia and Ding (2019) was constructed based on a public Chinese emotion corpus Gui et al. (2016) from the SINA NEWS website 555https://news.sina.com.cn/. The dataset contains 1,945 documents and 28,727 clauses. The number of candidate clause pairs is 490,367 and the number of valid emotion-cause clause pairs is 2,167.

In the experiment, we use the two styles of data split:

  • 10-fold cross validation. Selecting 90% of the data for training and the remaining 10% for testing stochastically (as same as Xia and Ding (2019)). The most previous works use the data split style.

  • Training/Validation/Test data set. Selecting 80% of the data for training, 10% of the data for validating and the remaining 10% for testing stochastically (as same as Fan et al. (2020a)). The data split style is more plausible than the first style.

Furthermore, when we extract the emotion-cause pairs, we obtain the emotions and causes simultaneously. Thus, the performance of emotion extraction and cause extraction were also evaluated. The precision P, recall R and F1 score defined in Gui et al. (2016); Xia and Ding (2019) are used to evaluate the performance of the three tasks.

4.2 Experimental Settings

We used the BERT

as our encoding backbone. During training, the AdamW optimizer with a weight decay of 0.01 was used for online learning, and the initial learning rate and warmup rate are set to 1e-5 and 0.1 respectively. The batch size is set to 2. As for regularization, dropout is applied for networks and the dropout rate is set to 0.1. We trained the model 20 epochs in total and adopted early stopping strategy. In the inference stage,

and the threshold was set to 0.5. The model was run on a Tesla V100 GPU.

4.3 Baselines

To demonstrate the effectiveness of our method, we compare our model with the following BERT baselines.

  • SL-NTS Yuan et al. (2020) regards the ECPE task as a sequence labeling problem and proposes a novel tagging scheme.

  • TransDGC Fan et al. (2020a) proposes a transition-based model to transform the task into a parsing-like directed graph construction procedure. 666It should be noted that the TransDGC method use the second data split style, while others methods use the first data split style.

  • ECPE-2D Ding et al. (2020a) uses a 2D transformers to model the interactions of different emotion-cause pairs.

  • PairGCN Chen et al. (2020b) constructs a pair graph convolutional network to model dependency relations among local neighboring candidate pairs.

  • RANKCP Wei et al. (2020) uses a graph attention network to model interactions between clauses and selects emotion-cause pairs by the ranking mechanism.

  • ECPE-MLL Ding et al. (2020b) extracts emotion-cause pairs based on sliding window multi-label learning. It is a state-of-the-art model of baselines.

Natural QL Pseudo QL Structured QL
Is it an emotion clause? emotion? emotion:_;cause:None
Is it a cause clause corresponding to ? ;cause? emotion:;cause:_
Is it an emotion clause corresponding to ? ;emotion? emotion:_;cause:
MM-R 80.62 (%F1) 80.51 (%F1) 79.72 (%F1)
Table 2: The performance of different query language designs (Natural, Pseudo and Structured QL) on ECPE task. “QL" denotes “Query Language". and are static emotion query, dynamic cause query and dynamic emotion query, respectively.

4.4 Variants

To show the effectiveness of the multi-turn structure and the rethink mechanism in the proposed method, We designed the following variants:

  • MRC-E2E is a single-turn MRC framework with an End-to-End style. The static queries is used to extract an emotion-cause pair set.

  • MM is a simple Multi-turn MRC framework. Compared with MM-R, it removes the rethink mechanism.

  • MM-D is proposed inspired by Chen et al. (2021); Mao et al. (2021), which is a Multi-turn MRC framework with Dual structure. One direction sequentially recognizes emotions and causes to obtain the candidate emotion-cause pair set with the help of static emotion query and dynamic cause query . In contrast, the other direction identifies causes first and then emotions to obtain another candidate set with the help of static cause query and dynamic emotion query . Finally, we take the intersection of the two candidate sets.

4.5 Main Results

Table 1 gives the comparative results for the ECPE task and the two sub-tasks. The proposed MM-R shows a clear advantage over other baselines, obtaining improvements of 6.10%, 3.13%, and 5.05%, respectively, over the previous baselines on the three tasks.

Specifically, for the ECPE task, the MM-R obtains 5.18%, 6.92% and 6.10% improvements in the , , and measures compared to ECPE-MLL. For the emotion extraction task, the MM-R performs 3.13% better than the best-performed baseline RANKCP, and it is worth noting that the increase in precision contributed most to the boost in the score. We believe that the high precision is due to the rethink mechanism filtering out some negative samples. The cause extraction task is more difficult because the cause clauses depend heavily on emotions. However, with the help of dynamic cause queries constructed by emotion semantic information, substantial increases (+5.05% ) are obtained compared to ECPE-MLL.

Besides, when we use the data split as the TransDGC Fan et al. (2020a), our method MM-R (Val) still obtains the state-of-the-art performance over the previous baselines.

Figure 3: An example in the test set. The emotion clause set {c3, c5} was obtained in the first turn and the candidate emotion-cause pair set {(c3, c4), (c5, c4)} in the second turn. After using the rethink mechanism, the valid emotion-cause pair set was identified as {(c5, c4)}.

5 Further Analysis

5.1 Effect of the Multi-turn Structure

MRC-E2E has the same label sparsity problem as pair-level end-to-end baselines, and it cannot model the interaction of emotions and causes due to its single-turn structure. However, MM, the most straightforward multi-turn framework among these similar methods, achieves 2.68%, 0.02%, and 2.84% improvements over MRC-E2E as shown in Table 3, which demonstrates the effectiveness of the multi-turn structure.

In addition, compared with end-to-end baseline ECPE-MLL, MRC-E2E still achieves 1.48%, 1.62% and 0.83% improvements of in three tasks, which demonstrates that the explicit emotion/cause semantic information played an important role.

Model Extraction of. (F1 %)
Emotion Cause E-C pair
MRC-E2E 90.34 77.92 75.35
MM 93.02 77.94 78.19
MM-D 93.67 79.47 78.76
MM-R 93.70 81.35 80.62
Table 3: Performance of variants.

5.2 Effect of the Rethink Mechanism

To verify the advantages of the rethink mechanism, MM-R was compared with the other two models (MM and MM-D). Specifically, compared with MM, MM-R achieved 0.68%, 3.41%, and 2.43% F1 improvements on the three tasks, demonstrating the the rethink mechanism’s effectiveness. Furthermore, MM-D achieved excellent performance with its dual structure. However, its score was 1.86% lower than MM-R in the ECPE task. The dual structure treats both emotion extraction and causes extraction equally, whereas the rethink mechanism is more trusting of emotion extraction. MM-R obtains a better performance due to emotion extraction being more reliable than cause extraction777We have illustrated in the introduction that emotion extraction is more reliable than cause extraction..

5.3 Effect of the Query Design

We explore the effect of different query language design on the model. As shown in Table 2, their F1 scores all exceeded 79% on the ECPE task. Concretely, Natural QL performs best in the ECPE task. Compared with Pseudo and Structured QL, Natural QL makes it easier for our model to understand the meaning of the queries, because BERT’s pre-trained corpus are all natural languages. The Pseudo QL can be regarded as a concise expression of Natural QL. Hence our model can still understand the meaning of queries. Structured QL is the most abstract of three QLs, transforming the three queries () into a unified paradigm. It gains the lowest F1 score, which indicates that the inductive ability of our model is not enough to understand abstract structured query language.

5.4 Case Analysis

Figure 3 shows the inference process of MM-R and the advantage of the rethink mechanism through an example in the test set. The candidate emotion-cause pair set {(c3, c4), (c5, c4)} were obtained in the first two turns, and their probabilities were and . After this, the proposed model adjusted the probabilities by rethinking whether each candidate emotion-cause pair was valid. The bottom of Figure 3 shows that the adjusted probabilities were and , and the threshold value was 0.5. This means that the answer to the dynamic emotion query “Is it an emotion clause corresponding to ‘but we are too poor to buy meat’?" was no for clause in the third turn. Therefore, candidate emotion-cause pair (c3, c4) was filtered out, while (c5, c4) was preserved.

5.5 Results for Emotion Cause Extraction

The proposed framework is also applied to the emotion cause extraction (ECE) task. Compared with the ECPE task, the ECE task has emotion annotations, and therefore in the MM-R framework, the emotion extraction of the first turn is omitted, and the emotion annotations are used to extract the corresponding cause clauses directly in the second turn. Finally, each candidate emotion-cause pair was verified by the rethink mechanism.

Comparisons are also conducted with recently proposed methods for the ECE task: RTHN Xia et al. (2019), RHNN Fan et al. (2019), KAG Yan et al. (2021) and 2-step RANKING Xu et al. (2021). Table 4 clearly demonstrates that our proposed framework achieves the state-of-the-art performance on the ECE task.

Methods Emotion Cause Extraction
P(%) R(%) F1(%)
RTHN 76.97 76.62 76.77
KAG 79.12 75.81 77.43
RHNN 81.12 77.25 79.14
2-step RANKING 80.76 78.45 79.59
MM-R 83.59 83.47 83.48
Table 4: Results on the Emotion Cause Extraction task.

6 Conclusions

This study transforms the emotion-cause pair extraction (ECPE) task into the machine reading comprehension (MRC) task and proposes a multi-turn MRC with rethink mechanism (MM-R). This structure, which extracts emotions and causes in turn, avoids the label sparsity problem and models the complicated corresponding relations between emotions and causes. In every turn, explicit semantic information can be used effectively. Furthermore, the rethink mechanism verifies each candidate emotion-cause pair by modeling the flow of information from causes to emotions. Experimental results on the ECPE corpus demonstrated the effectiveness of the proposed model.

Acknowledgements

This work was supported by National Key Research and Development Program of China (Grant No. 2020YFC0833402), National Natural Science Foundation of China (Grant Nos. 61976021, U1811262), and Beijing Academy of Artificial Intelligence (BAAI).

References

  • Bottou (2010) Léon Bottou. 2010.

    Large-scale machine learning with stochastic gradient descent.

    In Proceedings of COMPSTAT’2010, pages 177–186. Springer.
  • Chen et al. (2021) Shaowei Chen, Yu Wang, Jie Liu, and Yuelin Wang. 2021. Bidirectional machine reading comprehension for aspect sentiment triplet extraction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 12666–12674.
  • Chen et al. (2020a) Xinhong Chen, Qing Li, and Jianping Wang. 2020a. A unified sequence labeling model for emotion cause pair extraction. In Proceedings of the 28th International Conference on Computational Linguistics, pages 208–218.
  • Chen et al. (2020b) Ying Chen, Wenjun Hou, Shoushan Li, Caicong Wu, and Xiaoqiang Zhang. 2020b. End-to-end emotion-cause pair extraction with graph convolutional network. In Proceedings of the 28th International Conference on Computational Linguistics, pages 198–207.
  • Cheng et al. (2020) Zifeng Cheng, Zhiwei Jiang, Yafeng Yin, Hua Yu, and Qing Gu. 2020. A symmetric local search network for emotion-cause pair extraction. In Proceedings of the 28th International Conference on Computational Linguistics, pages 139–149.
  • Clark et al. (2019) Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. Boolq: Exploring the surprising difficulty of natural yes/no questions. arXiv preprint arXiv:1905.10044.
  • Cui and Zhang (2019) Leyang Cui and Yue Zhang. 2019. Hierarchically-refined label attention network for sequence labeling. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4115–4128.
  • Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186.
  • Ding et al. (2019) Zixiang Ding, Huihui He, Mengran Zhang, and Rui Xia. 2019. From independent prediction to reordered prediction: Integrating relative position and global label information to emotion cause identification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6343–6350.
  • Ding et al. (2020a) Zixiang Ding, Rui Xia, and Jianfei Yu. 2020a. Ecpe-2d: Emotion-cause pair extraction based on joint two-dimensional representation, interaction and prediction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3161–3170.
  • Ding et al. (2020b) Zixiang Ding, Rui Xia, and Jianfei Yu. 2020b. End-to-end emotion-cause pair extraction based on sliding window multi-label learning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3574–3583.
  • Fan et al. (2019) Chuang Fan, Hongyu Yan, Jiachen Du, Lin Gui, Lidong Bing, Min Yang, Ruifeng Xu, and Ruibin Mao. 2019. A knowledge regularized hierarchical approach for emotion cause analysis. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5614–5624.
  • Fan et al. (2020a) Chuang Fan, Chaofa Yuan, Jiachen Du, Lin Gui, Min Yang, and Ruifeng Xu. 2020a. Transition-based directed graph construction for emotion-cause pair extraction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3707–3717.
  • Fan et al. (2021) Chuang Fan, Chaofa Yuan, Lin Gui, Yue Zhang, and Ruifeng Xu. 2021. Multi-task sequence tagging for emotion-cause pair extraction via tag distribution refinement. IEEE/ACM Transactions on Audio, Speech, and Language Processing.
  • Fan et al. (2020b) Rui Fan, Yufan Wang, and Tingting He. 2020b. An end-to-end multi-task learning network with scope controller for emotion-cause pair extraction. In CCF International Conference on Natural Language Processing and Chinese Computing, pages 764–776. Springer.
  • Gui et al. (2017) Lin Gui, Jiannan Hu, Yulan He, Ruifeng Xu, Qin Lu, and Jiachen Du. 2017. A question answering approach for emotion cause extraction. In EMNLP.
  • Gui et al. (2016) Lin Gui, Dongyin Wu, Ruifeng Xu, Qin Lu, and Yu Zhou. 2016. Event-driven emotion cause extraction with corpus construction. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1639–1649.
  • Gui et al. (2018) Lin Gui, Ruifeng Xu, Dongyin Wu, Qin Lu, and Yu Zhou. 2018. Event-driven emotion cause extraction with corpus construction. In Social Media Content Analysis: Natural Language Processing and Beyond, pages 145–160. World Scientific.
  • Jia et al. (2020) Xiangyu Jia, Xinhai Chen, Qian Wan, and Jie Liu. 2020. A novel interactive recurrent attention network for emotion-cause pair extraction. In 2020 3rd International Conference on Algorithms, Computing and Artificial Intelligence, pages 1–9.
  • Li and Xu (2014) Weiyuan Li and Hua Xu. 2014. Text-based emotion classification using emotion cause extraction. Expert Systems with Applications, 41(4):1742–1749.
  • Li et al. (2018) Xiangju Li, Kaisong Song, Shi Feng, Daling Wang, and Yifei Zhang. 2018.

    A co-attention neural network model for emotion cause analysis with emotional context awareness.

    In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4752–4757.
  • Li et al. (2020) Xiaoya Li, Jingrong Feng, Yuxian Meng, Qinghong Han, Fei Wu, and Jiwei Li. 2020.

    A unified mrc framework for named entity recognition.

    In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5849–5859.
  • Li et al. (2019) Xiaoya Li, Fan Yin, Zijun Sun, Xiayu Li, Arianna Yuan, Duo Chai, Mingxin Zhou, and Jiwei Li. 2019. Entity-relation extraction as multi-turn question answering. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1340–1350.
  • Liu et al. (2020) Jian Liu, Yubo Chen, Kang Liu, Wei Bi, and Xiaojiang Liu. 2020. Event extraction as machine reading comprehension. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1641–1651.
  • Loshchilov and Hutter (2018a) Ilya Loshchilov and Frank Hutter. 2018a. Decoupled weight decay regularization. In International Conference on Learning Representations.
  • Loshchilov and Hutter (2018b) Ilya Loshchilov and Frank Hutter. 2018b. Fixing weight decay regularization in adam.
  • Mao et al. (2021) Yue Mao, Yi Shen, Chao Yu, and Longjun Cai. 2021. A joint training dual-mrc framework for aspect based sentiment analysis. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 13543–13551.
  • McCann et al. (2018) Bryan McCann, Nitish Shirish Keskar, Caiming Xiong, and Richard Socher. 2018. The natural language decathlon: Multitask learning as question answering. arXiv preprint arXiv:1806.08730.
  • Qin et al. (2021) Libo Qin, Tailu Liu, Wanxiang Che, Bingbing Kang, Sendong Zhao, and Ting Liu. 2021. A co-interactive transformer for joint slot filling and intent detection. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8193–8197. IEEE.
  • Rei and Søgaard (2019) Marek Rei and Anders Søgaard. 2019. Jointly learning to label sentences and tokens. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6916–6923.
  • Shan and Zhu (2020) Jingzhe Shan and Min Zhu. 2020. A new component of interactive multi-task network model for emotion-cause pair extraction. In Journal of Physics: Conference Series, volume 1693, page 012022. IOP Publishing.
  • Shen et al. (2021) Yongliang Shen, Xinyin Ma, Zeqi Tan, Shuai Zhang, Wen Wang, and Weiming Lu. 2021. Locate and label: A two-stage identifier for nested named entity recognition. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 2782–2794, Online.
  • Singh et al. (2021) Aaditya Singh, Shreeshail Hingane, Saim Wani, and Ashutosh Modi. 2021. An end-to-end network for emotion-cause pair extraction. arXiv preprint arXiv:2103.01544.
  • Song et al. (2020) Haolin Song, Chen Zhang, Qiuchi Li, and Dawei Song. 2020. End-to-end emotion-cause pair extraction via learning to link. arXiv preprint arXiv:2002.10710.
  • Sun et al. (2021) Qixuan Sun, Yaqi Yin, and Hong Yu. 2021. A dual-questioning attention network for emotion-cause pair extraction with context awareness. arXiv preprint arXiv:2104.07221.
  • Tang et al. (2020) Hao Tang, Donghong Ji, and Qiji Zhou. 2020.

    Joint multi-level attentional model for emotion detection and emotion-cause pair extraction.

    Neurocomputing, 409:329–340.
  • Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008.
  • Veličković et al. (2017) Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903.
  • Wei et al. (2020) Penghui Wei, Jiahao Zhao, and Wenji Mao. 2020. Effective inter-clause modeling for end-to-end emotion-cause pair extraction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3171–3181.
  • Wu et al. (2020) Sixing Wu, Fang Chen, Fangzhao Wu, Yongfeng Huang, and Xing Li. 2020. A multi-task learning neural network for emotion-cause pair extraction. In ECAI 2020, pages 2212–2219. IOS Press.
  • Xi et al. (2021) Xiangyu Xi, Wei Ye, Shikun Zhang, Quanxiu Wang, Huixing Jiang, and Wei Wu. 2021. Capturing event argument interaction via a bi-directional entity-level recurrent decoder. arXiv e-prints, pages arXiv–2107.
  • Xia and Ding (2019) Rui Xia and Zixiang Ding. 2019. Emotion-cause pair extraction: A new task to emotion analysis in texts. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1003–1012.
  • Xia et al. (2019) Rui Xia, Mengran Zhang, and Zixiang Ding. 2019. Rthn: A rnn-transformer hierarchical network for emotion cause extraction. arXiv preprint arXiv:1906.01236.
  • Xu et al. (2019) Bo Xu, Hongfei Lin, Yuan Lin, Yufeng Diao, Liang Yang, and Kan Xu. 2019. Extracting emotion causes using learning to rank methods from an information retrieval perspective. IEEE Access, 7:15573–15583.
  • Xu et al. (2021) Bo Xu, Hongfei Lin, Yuan Lin, and Kan Xu. 2021. Two-stage supervised ranking for emotion cause extraction. Knowledge-Based Systems, page 107225.
  • Yan et al. (2021) Hanqi Yan, Lin Gui, Gabriele Pergola, and Yulan He. 2021. Position bias mitigation: A knowledge-aware graph model for emotion cause extraction. arXiv preprint arXiv:2106.03518.
  • Yu et al. (2021) Jiaxin Yu, Wenyuan Liu, Yongjun He, and Chunyue Zhang. 2021. A mutually auxiliary multitask model with self-distillation for emotion-cause pair extraction. IEEE Access, 9:26811–26821.
  • Yu et al. (2019) Xinyi Yu, Wenge Rong, Zhuo Zhang, Yuanxin Ouyang, and Zhang Xiong. 2019. Multiple level hierarchical network-based clause selection for emotion cause extraction. IEEE Access, 7:9071–9079.
  • Yuan et al. (2020) Chaofa Yuan, Chuang Fan, Jianzhu Bao, and Ruifeng Xu. 2020. Emotion-cause pair extraction as sequence labeling based on a novel tagging scheme. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3568–3573.
  • Zheng et al. (2019) Yukun Zheng, Jiaxin Mao, Yiqun Liu, Zixin Ye, Min Zhang, and Shaoping Ma. 2019. Human behavior inspired machine reading comprehension. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 425–434.
  • Zhong and Chen (2021) Zexuan Zhong and Danqi Chen. 2021. A frustratingly easy approach for entity and relation extraction. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 50–61.
  • Zhou et al. (2021) Yang Zhou, Yubo Chen, Jun Zhao, Yin Wu, Jiexin Xu, and Jinlong Li. 2021. What the role is vs. what plays the role: Semi-supervised event argument extraction via dual question answering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 14638–14646.