End to End Chinese Lexical Fusion Recognition with Sememe Knowledge

04/11/2020 ∙ by Yijiang Liu, et al. ∙ Wuhan University 0

In this paper, we present Chinese lexical fusion recognition, a new task which could be regarded as one kind of coreference recognition. First, we introduce the task in detail, showing the relationship with coreference recognition and differences from the existing tasks. Second, we propose an end-to-end joint model for the task, which exploits the state-of-the-art BERT representations as encoder, and is further enhanced with the sememe knowledge from HowNet by graph attention networks. We manually annotate a benchmark dataset for the task and then conduct experiments on it. Results demonstrate that our joint model is effective and competitive for the task. Detailed analysis is offered for comprehensively understanding the new task and our proposed model.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 9

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Coreference is one important topic in linguistics [Gordon and Hendrick1998, Pinillos2011]

, and coreference recognition has been extensively researched in the natural language processing (NLP) community

[Ng and Cardie2002, Lee et al.2017, Qi et al.2012, Fei et al.2019]. There are different kinds of coreferences, such as pronoun anaphora and abbreviation [Mitkov1998, Ruslan Mitkov1999, Muñoz and Montoyo2002, Choubey and Huang2018]. Here we examine the phenomenon of Chinese lexical fusion, where two closely related words in a paragraph are united by a fusion form of the same meaning, and the fusion word can be seen as the coreference of the two separation words. Since the fusion words are always out-of-vocabulary (OOV) words in downstream paragraph-level tasks such as reading comprehension, summarization, machine translation and etc., which can hinder the overall understanding and lead to degraded performance, the new task can offer informative knowledge to these tasks.

[1][red] on line, colback = #1!50!white, colframe = #1!50!black, boxsep = 0pt, left = 1pt, right = 1pt, top = 2pt, bottom = 2pt, boxrule = 0pt, bottomrule = 0pt, toprule = 0pt

[1][red] on line, colback = #1!20!white, colframe = #1!50!black, boxsep = 0pt, left = 1pt, right = 1pt, top = 2pt, bottom = 2pt, boxrule = 0pt, bottomrule = 0pt, toprule = 0pt

卡纳瓦罗打破[red]受访惯例,公开训练未[blue]接受[blue]采访。
Cannavaro broke the convention of [red]accepting an [red]interview, did not [blue]accept an [blue]interview for public training.
首批医务人员昨日[red]返杭,其余人员预计两周内[blue]回到[blue]杭州。
The first medical personnel [red]returned to [red]Hangzhou yesterday, the others are expected to [blue]return to [blue]Hangzhou within two weeks.
央行宣布[blue]下调人民币贷款基准[blue]利率,这是央行近期进行的第二次[red]降息调整。
The central bank announced to [blue]reduce the benchmark [blue]interest rate for RMB loans, which is the second time to [red]reduce the interest rate recently.
Table 1: Examples of Chinese lexical fusion.

Table 1 shows three examples of Chinese lexical fusion. In the examples, “受访”(accept an interview), “返杭”(returned to Hangzhou) and “降息”(reduce the interest rate) are the fusion words, and their coreferences are “接受(accept)/采访(interview)”, “回到(return)/杭州(Hangzhou)”, and “下调 (reduce)/利率(interest rate)”, respectively. Each corresponding of the fusion word consists of two separation words. Besides, each fusion word character is semantically corresponding to one of the separation words, which can be regarded as fine-grained coreferences. As shown, we have six fine-grained coreferences by the three example fusion words: “受接受(accept)”, “访采访 (interview)”, “返回到(return)”, “杭杭州(Hangzhou)”, “降下调 (reduce)” and “息利率(interest rate)”.

Lexical fusion is used frequently in the Chinese language. Moreover, the fusion words are usually rarer words compared with their separation words coreferred, which are more difficult to be handled by NLP models[Zhang and Yang2018, Gui et al.2019]. Luckily, the meaning of the fusion words can be derived from that of the separation words. Thus recognizing the lexical fusion would be beneficial for downstream paragraph (or document)-level NLP applications such as machine translation, information extraction, summarization, etc.[Li and Yarowsky2008, Ferreira et al.2013, Kundu et al.2018]. For example, for deep semantic parsing or translation, the fusion words “受访”(UNK) can be substituted directly by the separation words “接受”(accept) and “访问”(interview), as the same fusion words are rarely occurred in other paragraphs.

The recognition of lexical fusion can be accomplished by two subtasks. Given one paragraph, the fusion words, as well as the separation words should be detected as the first step, which is referred to as mention detection. Second, coreference recognition is performed over the detected mentions, linking each character in the fusion words to their coreferences, respectively. By the second step, full lexical fusion coreferences are also recognized concurrently. The two steps can be conducted jointly in a single end-to-end model [Lee et al.2017], which helps avoid the error propagation problem, and meanwhile, enable the two subtasks with full interaction.

In this paper, we present a competitive end-to-end model for lexical fusion recognition. Contextual BERT representations [Devlin et al.2019] are adopted as encoder inputs as they have achieved great success in a number of NLP tasks [Tian et al.2019, Zhou et al.2019, Xu et al.2019]. For mention detection, a CRF decoder [Huang et al.2015] is exploited to detect all mention words, including both the fusion and the separation words. Further, we use a BiAffine decoder for coreference recognition [Zhang et al.2017, Bekoulis et al.2018], determining a given mention pair either to be a coreference or not.

Since our task is semantic oriented, we use the sememe knowledge provided in HowNet[Dong and Dong2003] to help capturing the semantic similarity between the characters and the separation words. HowNet has achieved success in many Chinese NLP tasks in recent years [Duan et al.2007, Gu et al.2018, Ding et al.2019, Li et al.2019]. Both Chinese characters and words are defined by senses of sememe graphs in it, and we exploit graph attention networks (GAT) [Velickovic et al.2018] to model the sememe graphs to enhance our encoder.

Finally, we manually construct a high-quality dataset to evaluate our models. The dataset consists of 7,271 cases of the lexical fusion, which are all exploited as the test instances. To train our proposed models, we construct a pseudo dataset automatically from the web resource. Experimental results show that the auto-constructed training dataset is highly effective for our task. The end-to-end models achieved better performance than the pipeline models, and meanwhile the sememe knowledge can also bring significant improvements for both the pipeline and end-to-end models. Our final model can obtain an F-measure of 79.64% for lexical fusion recognition. Further, we conduct in-depth analysis work to understand the proposed task and our proposed models.

In summary, we make the following contributions in this paper:

  • We introduce a new task of lexical fusion recognition, providing a gold-standard test dataset and an auto-constructed pseudo training dataset for the task, which can be used as a benchmark for future research.

  • We propose a competitive end-to-end model for lexical fusion recognition task, which helps to integrate the mention recognition with coreference identification based on BERT representations.

  • We make use of the sememe knowledge from HowNet to help capturing the semantic relevance between the characters in the fusion form and the separation words.

All the codes and datasets will be open-sourced at

https://github.com/xxx under Apache License 2.0.

2 Chinese Lexical Fusion

Lexical fusion refers to the phenomenon that a composition word semantically corresponds with related words nearby in the contexts. The composition word can be seen as a compact form of the separation words in the paragraph, where the composition word is referred to as the fusion form of the separation words. The fusion form is often an OOV word, which brings difficulties in understanding of a given paragraph. We can use a tuple to define the phenomenon in a paragraph: , where denotes the fusion word, and are the separation words. We regard the as a coreference of . More detailedly, the number equals to the length of word , and the th character of word corresponds to the separation word as one fine-grained coreference. The task is highly semantic-oriented, as all the fine-grained coreferences are semantically related.

Notice that Chinese lexical fusion has the following attributes, making it different from others:

  • Each character of word must correspond to one and only one separation word. The one-one mappings (clusters) should be strictly abided by. For example, 二圣 (two saints), 李治 (Zhi Li), 武则天 (Zetian Wu) 111The last two words of the tuple refer to names of two distinguished person. is not a lexical fusion phenomenon, because both the two characters “二” (two) and “圣” (saint) can not associate with any of the separations.

  • For each fine-grained coreference, the th character of word is mostly borrowed from its separated coreference directly, but it is not always true. A semantically-equaled character could be used as well, which greatly increases the difficulty of Chinese lexical fusion recognition. The given examples in Table 1 demonstrate the rule. As shown, the coreferences “返回到(return)”, “降下调 (reduce)” and “息利率(interest rate)” are all of these cases.

  • The separation words are semantically independent, especially not indicating a single specific entity. Thus the fine-grained coreferences are semantically different in lexical fusion. For example, 北大 (Peking University), 北京 (Peking), 大学 (University) , the semantic meaning of “北京 (Peking) 大学 (University)” is inseparable.

  • The Chinese lexical fusion phenomenon could be either forward or backward. Table 1 shows two examples of backward references (i.e., “受访接受/采访 (accept an interview)” and “返杭回到/杭州 (returned to Hangzhou)”) and one example of forward reference (i.e., “降息下调/利率 (reduce the interest rate)”).

By the above characters, we can easily differentiate the Chinese lexical fusion with several other closely-related linguistic phenomena, as an example, the abbreviation, the illustrated negative examples in item (1) and (3) are both abbreviations.

For simplicity, we limit our focus on in this work, because this situation occupies over 99% of all the lexical fusion cases according to our preliminary statistics. Thus we can call tuples as triples all though this paper.

3 Method

We built an end-to-end model for the lexical fusion recognition task. The recognition of lexical fusion consists of two steps. First mention detection is performed to collect all mention words, including the fusion words and their separations. By applying the BIO tagging scheme associating with mention types, this subtask can be converted into a typical sequence labeling task [Ohta et al.2012, Ma and Hovy2016]. Second, we conduct pair-wise character-word clustering, obtaining one-one fine-grained coreferences and reformatting them to triples of . The end-to-end model alleviates the problem of error propagation in pipeline methods through a joint way. The model consists of an encoder, a CRF decoder for mention recognition and a biaffine decoder for coreference recognition. Further, we enhance the encoder with sememe knowledge from HowNet.

3.1 Basic Model

Encoder

We exploit the BERT as the basic encoder because it has achieved state-of-the-art performances for a number of NLP tasks [Devlin et al.2019]. Given a paragraph the output of BERT is a sequence of contextual representation at the character-level:

(1)

where is our desired representation.

Mention Detection

The CRF decoder is exploited to obtain the sequence labeling outputs. First, the encoder outputs are transformed into tag scores at each position:

(2)

where , is the number of tag classes. Then for each tag sequence

, its probability can be computed by:

(3)

where is a model parameter to indicate output tag transition score and is a normalization score. We can use standard Viterbi to obtain the highest-probability tagging sequence.

Coreference Recognition

We take the input character representation obtained by the encoder as well as the output tag sequence of mention detection as inputs. For the output tag sequence, we exploit a simple embedding matrix

to convert tags into vectors

. Then we concatenate the two kinds of representations, getting the encoder output , where . We treat coreference recognition as a binary classification problem. Given the encoder output and two positions and , we judge whether the relation between the two characters and is a coreference or not. For one fine-grained coreference and , we regard all characters in be connected to .

The score of the binary classification is computed by a simple BiAffine operation,

(4)

where is one two-dim vector. We can refer to [Dozat and Manning2017] for the details of the BiAffine operation, which has shown strong capabilities in similar tasks [Zhang et al.2018].

Training

For mention detection, the supervised training objective is to minimize the cross-entropy of the gold-standard tagging sequence:

(5)

where is the supervised answer.

For coreference recognition, we adopt averaged cross-entropy loss overall input pairs as the training objective:

(6)

where denotes the ground-truth relation. The probability is computed by:

(7)

where is a normalization factor.

We combine losses from the two subtasks together for joint training:

(8)

where is a hyper-parameter to balance the losses of the two subtasks.

Figure 1: Overview of the end-to-end architecture for lexical fusion recognition.

3.2 Sememe-Enhanced Encoder

Sememes and HowNet

Sememe is regarded as the minimum semantic unit for the Chinese language, which has been exploited for several semantic-oriented tasks, such as word sense disambiguation, event extraction and relation extraction [Gu et al.2018, Ding et al.2019, Li et al.2019]. Our task is also semantic oriented because the lexical fusion and coreference are both semantic related. Thus sememe should be one kind of useful information for our model.

Figure 2: The architecture of sememe, sense and word in HowNet, where a sememe may be shared across senses, eg. the “Subtract”.

We follow the majority of the previous work, extracting sememes for characters from HowNet, a manually-constructed sememe knowledge base. HowNet defines over 118,000 words (including characters) using about 2,000 sememes.222Chinese characters are also basic semantic units for meaning expressing like full words. Figure 2 illustrates the annotations in HowNet. We can see that each word is associated with several senses, and further, each sense is annotated with several sememes, where sememes are organized by graphs.

Sememe to Character Representation

For each character , we make use of all possible HowNet words covering , as shown in Figure 3, and further extract all the included senses by these words. Each sense corresponds to one sememe graph, as shown in Figure 2. The sememe to character representation is obtained by two steps. First, we obtain the sense representation by its sememe graph and the position offset of its source word. Then, we aggregate all sense representations to reach a character-level representation, resulting in the sememe-enhanced encoder.

Figure 3: The matched word and relative position for each character in the sequence.

We use standard GAT to represent the sememe graph. Let denote all the sememes belonging to a given sense , and their internal graph structure is denoted by , after apply the GAT module, we can get:

(9)

where indicates the embedding operation.

Further, we obtain the first representation part of by averaging over . The second part is obtained straightforwardly by the embedding of the position offset of the sense’s source word. The position offset is denoted by , where and indicate the relative position of the start and end characters of the source word to the current character, which has been illustrated in Figure 3. We use the position offset as a single unit for embedding. Following, we concatenate the two parts, resulting in the sense representation:

(10)

where denotes vector concatenation.

Finally, we aggregate all sense representations by global attention [Luong et al.2015] with the guide of the BERT outputs to obtain character-level representations. Let denote the set of sense representations for one character , the sememe-enhanced representation for character can be computed by:

(11)

where is a model parameter for attention calculation, and are the desired outputs which is used instead of the BERT outputs for decoder.

4 Experiments

4.1 Dataset

Test Data

We build a lexical fusion dataset manually in this work, where the raw corpus is collected from SogouCA [Wang et al.2008], a news corpus of 18 channels, including domestic, international, sports, social, entertainment, etc. The BRAT Rapid Annotation Tool [Stenetorp et al.2012] is used for annotation. We label the boundary of mentions in the paragraph, determine their categories (fusion or separation words), link characters in the fusion words to the referred separation words, and finally format the annotation results as triples.

We annotate 17,000 paragraphs by five annotators who major in Chinese linguistics. After removing the paragraphs without lexical fusion, the five annotators check the annotations once again to ensure the quality. Finally, 7,271 lexical fusion cases are obtained with 91% consensus.

Pseudo Training Dataset

We construct a pseudo lexical fusion corpus to train the models, by making use of an online lexicon where words are offered with explanatory notes.

333https://cidian.911cha.com/ For each two-character word, we check if it can be split into two separation words. If successful, we obtain one context-independent triple. Finally, we collect 1,608 well-formed triples of lexical fusions and treat them as seeds to construct pseudo training instances. Note that the fusion words of these triples are currently acceptable and widely used by users, such as “停车(parking vehicles)停放(parking)/车辆(vehicle)”. Then, we search for the paragraphs containing all three words of a certain triple, regarding them as valid cases of Chinese lexical fusion[Mintz et al.2009]. Finally, we obtain 11,034 paragraphs, which are divided into training and development sections for model parameter learning and hyper-parameter tuning, respectively.

Table 2 summarizes the overall data statistics of the training, development and testing sections, including the number of cases (lexical fusion), the averaged paragraph length (by character count) and the number of unique triples, respectively.

#Case #U.Triple #A.Length #N.Case #N.U.Triple
Training 7,400 1,531 106 - -
Development 2,213 1,001 105 94 77
Testing 7,271 1,661 91 3959 957
Table 2: Data statistics of our corpus, where A. , U. and N. indicate average, unique and new, respectively.

4.2 Evaluation

A triple is regarded as correct if all the three elements are correctly recognized and meanwhile on their exact positions, We calculate triple-level and fine-grained precision (P), recall (R) and F-measure (F) values, and adopt the triple-level values as the major metrics to evaluate the model performance. We also calculate mention-level P, R and F values to evaluate the performance of mention detection.

4.3 Settings

All the hyper-parameters are determined according to the performance on the development dataset. We exploit the pretrained basic BERT representations as inputs for encoder [Devlin et al.2019].444https://github.com/google-research/bert The tag, sememe and position embedding sizes are set to 25, 200 and 12, respectively. The head number of GAT is 3. We exploit dropout with a ration of 0.2 on the encoder outputs to avoid overfitting [Srivastava et al.2014], and optimize all model parameters (including the BERT part) by standard back-propagation using Adam [Kingma and Ba2015] with a learning rate .

          Model Entity Detection Coreference Recognition
Fusion Separation Fine-Grained Triple-Level
Entity Detection Coreference Recognition
P R F P R F P R F P R F
Pipeline
basic 83.10 83.88 83.49 82.84 84.46 83.64 76.09 76.66 76.37 74.26 75.23 74.74
GAT(word,Hownet) 84.57 83.02 83.79 86.92 81.79 84.28 85.07 74.27 79.30 85.02 73.27 78.71
Joint
basic 82.85 81.40 82.12 85.79 82.46 84.09 81.65 74.32 77.81 80.32 72.94 76.45
GAT(char,pseudo) 84.61 83.15 83.87 86.83 82.58 84.56 83.19 75.81 79.33 81.57 74.46 77.85
GAT(char,Hownet) 84.79 84.27 84.53 88.07 81.75 84.79 84.71 75.15 79.64 83.72 73.87 78.49
GAT(word,pseudo) 84.55 83.69 84.12 85.70 84.82 85.26 87.88 73.12 79.82 89.38 70.62 78.90
GAT(word,Hownet) 85.18 84.16 84.67 88.64 83.98 86.25 85.63 75.60 80.30 86.19 74.02 79.64
Table 3: Final results on the test dataset, where basic indicates no sememe information is used.

4.4 Results

Table 3 shows the final results on the test dataset. Aiming to investigate the influence of sememe structure in detail, we construct a pseudo graph structure for comparison where all sememes inside a sense are mutually connected. In addition, we conduct comparisons by using only the sememes from characters (single-character words), in order to explore the effect of the word-level sememe information. Our final model, the end-to-end model with word-level sememe-enhanced encoder (i.e., joint + GAT(word_real)), achieves competitive performance, where the triple-level F-measure reaches , which is the best-performance model, significantly better than the basic model without using the sememe information.555

The p-value is below 0.0001 by using pairwise t-test.

In addition to the contents of 3, we also tried the pipeline approach as a contrast. The triple-level F-measure of pipeline models with or without word-level sememe graph are 78.71% and 74.74%, respectively.

According to the results, our proposed sememe-enhanced encoder is effective, bringing significant improvements over the corresponding basic model. The improvements on the triple-level F-measure is . The performance differences of our sememe-enhanced encoder by using the pseudo sememe graph and the real sememe graph indicate the effectiveness of sememe structure. The real sememe graph can give consistently better performance (i.e., an increase of on average) than the pseudo graph. As shown, the word-level sememe structure can obtain F-measure improvements of , indicating the usefulness of the word-level information.

4.5 Analysis

Influence of Lexical Fusion Types

We investigate the model performance with respect to different lexical fusion types. We classify the fusion word types by IV/OOV according to the training corpus, and further differentiate a fine-grained coreference by whether the fusion character is borrowed from its separation word (denoted by A) or not (denoted by B). Table

4 shows the results. Our models perform better for the IV categories than the OOV, which confirms with our intuition. In addition, we divide the IV/OOV further into AA and AB categories.666We ignore the BB categories as the number is below 10. We can find that AB is much more difficult, obtaining only 41.1 of the F1 score. Further, by examining the overall performance of fine-grained coreference of type A and B, we can see that Type B leads to the low performance mainly. The performance gap between the two types is close to 50 on the F1 score.

Model IV OOV Fine-Grained
AB AA AB AA B A
Joint 86.4 94.5 30.4 81.8 28.1 82.4
GAT(word_real) 89.5 94.7 41.1 82.9 36.5 83.7
Table 4: F1 scores by different lexical fusion types.

Effect of Sememe Information

In order to understand the sememe-enhanced encoder in-depth, we examine the sense-level attention weights by an example. As shown in Figure 4, a fine-grained coreference “降下调 (reduce)” is used for illustration, where the three characters “下”(lower), “调”(adjust) and “降”(reduce) are studied. Each character includes a set of senses, which are listed by the squares.777Senses with very low weights are filtered. We can see that senses with shared sememes can mutually enhance their respective attention weights, which is consistent with the goal of coreference recognition. It is difficult to establish such a connection without using sememes.

Figure 4: The sense-level attention weights of “降下调 (reduce)”, where the colored squares represent senses, the senses with shared sememes are linked with double arrows.

Impact of Order of Mentions

Figure 5(a) shows the F-measure values according to the relative order of the mentions. Intuitively, the recognition of the forward references is more difficult than that of the backward references. The results confirm our intuition, where the F1 score of backward reference is 4.2 points higher on average. In addition, our final model can improve performance of both types significantly.

(a) The triple-level F1 scores in terms of mention order.
(b) The triple-level F1 scores with respect to the distance between the separation words.
Figure 5: Impact of the order and the distance.

Impact of the Separation Word Distance

The distance between the two separation words should be an important factor in coreference recognition. Intuitively, as the distance increases, the difficulty should be also increased greatly. Here we conduct analysis to verify this intuition. Figure 5(b) shows the comparison results, which is consistent with our supposition. In addition, we can see that our final end-to-end model behaves much better, with relatively smaller decreases as the distance increases.

5 Related Work

Coreference recognition has been investigated extensively in NLP for decades [McCarthy and Lehnert1995, Cardie and Wagstaff1999, Ng and Cardie2002, Elango2005]. Lexical fusion can be regarded as one kind of coreference, however, it has received little attention in the NLP community. Our proposed models are inspired by the work of neural coreference resolution [Fernández-Gallego et al.2016, Clark and Manning2016, Xu et al.2017, Lee et al.2017, Zhang et al.2018]. We adapt these models by considering task-specific features of Chinese lexical fusion, for example, enhancing the encoder with a GAT module for structural sememe information.

Another most closely-related topic is abbreviation [Zhong1985]. There have been several studies on abbreviation prediction, recovery and dictionary construction [Sun et al.2008, Li and Yarowsky2008, Liu et al.2009, Zhang and Sun2018]. Lexical fusion is different from abbreviation in many points. For example, abbreviation always refers to one inseparable mention, which is not necessary for lexical fusion. Besides, lexical fusion should abide by certain word construction rules, while abbreviation is for free.

BERT and its variations have achieved the leading performances for GLUE benchmark datasets [Devlin et al.2019, Liu et al.2019a, Liu et al.2019b]. For the close tasks such as coreference resolution and relation extraction, BERT representations have also shown competitive results [Joshi et al.2019, Lin et al.2019], which inspires our work by using it as basic inputs aiming for competitive performance.

The sense and sememe information has been demonstrated effective for several semantic-oriented NLP tasks [Niu et al.2017, Gu et al.2018, Zeng et al.2018, Ding et al.2019, Qi et al.2019]. HowNet offers a large knowledge base of sememe-based [Dong and Dong2003], which has been adopted for sememe extraction. We encode the sememes by the form of a graph naturally, and then exploit GAT to enhance our task encoder.

6 Conclusion

In this work, we introduced the task of lexical fusion recognition in Chinese and then presented an end-to-end model for the new task. BERT representation was exploited as the basic input for the models, and the model is further enhanced with the sememe knowledge from HowNet by graph attention networks. We manually annotated a benchmark dataset for the task, which was used to evaluate the models. Experimental results on the annotated dataset indicate the competitive performance of our final model, and the effectiveness of the joint modeling and the sememe-enhanced encoder. Analysis is offered to understand the new task and the proposed model in-depth.

References

  • [Bekoulis et al.2018] Giannis Bekoulis, Johannes Deleu, Thomas Demeester, and Chris Develder. 2018. Joint entity recognition and relation extraction as a multi-head selection problem. Expert Systems with Applications, 114:34–45.
  • [Cardie and Wagstaff1999] Claire Cardie and Kiri Wagstaff. 1999. Noun phrase coreference as clustering. In 1999 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora.
  • [Choubey and Huang2018] Prafulla Kumar Choubey and Ruihong Huang. 2018. Improving event coreference resolution by modeling correlations between event coreference chains and document topic structures. In Proceedings of ACL, pages 485–495, Melbourne, Australia, July.
  • [Clark and Manning2016] Kevin Clark and Christopher D. Manning. 2016.

    Improving coreference resolution by learning entity-level distributed representations.

    In Proceedings of ACL, pages 643–653, Berlin, Germany, August.
  • [Devlin et al.2019] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL, pages 4171–4186.
  • [Ding et al.2019] Ning Ding, Ziran Li, Zhiyuan Liu, Haitao Zheng, and Zibo Lin. 2019.

    Event detection with trigger-aware lattice neural network.

    In Proceedings of EMNLP-IJCNLP, pages 347–356, Hong Kong, China, November.
  • [Dong and Dong2003] Zhendong Dong and Qiang Dong. 2003. Hownet-a hybrid language and knowledge resource. In Proceedings of NLPKE, pages 820–824. IEEE.
  • [Dozat and Manning2017] Timothy Dozat and Christopher D. Manning. 2017. Deep biaffine attention for neural dependency parsing. In Proceedings of ICLR.
  • [Duan et al.2007] Xiangyu Duan, Jun Zhao, and Bo Xu. 2007. Word sense disambiguation through sememe labeling. In Proceedings of IJCAI, pages 1594–1599.
  • [Elango2005] Pradheep Elango. 2005. Coreference resolution: A survey. University of Wisconsin, Madison, WI.
  • [Fei et al.2019] Hongliang Fei, Xu Li, Dingcheng Li, and Ping Li. 2019.

    End-to-end deep reinforcement learning based coreference resolution.

    In Proceedings of ACL, pages 660–665, Florence, Italy, July.
  • [Fernández-Gallego et al.2016] María Pilar Fernández-Gallego, Álvaro Mesa-Castellanos, Alicia Lozano-Díez, and Doroteo T. Toledano. 2016. Detection of publicity mentions in broadcast radio: Preliminary results. In Alberto Abad, Alfonso Ortega, António Teixeira, Carmen García Mateo, Carlos D. Martínez Hinarejos, Fernando Perdigão, Fernando Batista, and Nuno Mamede, editors, Advances in Speech and Language Technologies for Iberian Languages, pages 108–116, Cham. Springer International Publishing.
  • [Ferreira et al.2013] Rafael Ferreira, Luciano de Souza Cabral, Rafael Dueire Lins, Gabriel Pereira e Silva, Fred Freitas, George DC Cavalcanti, Rinaldo Lima, Steven J Simske, and Luciano Favaro. 2013.

    Assessing sentence scoring techniques for extractive text summarization.

    Expert systems with applications, 40(14):5755–5764.
  • [Gordon and Hendrick1998] Peter C Gordon and Randall Hendrick. 1998. The representation and processing of coreference in discourse. Cognitive science, 22(4):389–424.
  • [Gu et al.2018] Yihong Gu, Jun Yan, Hao Zhu, Zhiyuan Liu, Ruobing Xie, Maosong Sun, Fen Lin, and Leyu Lin. 2018. Language modeling with sparse product of sememe experts. In Proceedings of EMNLP, pages 4642–4651, Brussels, Belgium, October-November.
  • [Gui et al.2019] Tao Gui, Ruotian Ma, Qi Zhang, Lujun Zhao, Yu-Gang Jiang, and Xuanjing Huang. 2019. Cnn-based chinese NER with lexicon rethinking. In Proceedings of IJCAI, pages 4982–4988.
  • [Huang et al.2015] Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional lstm-crf models for sequence tagging. arXiv preprint arXiv:1508.01991.
  • [Joshi et al.2019] Mandar Joshi, Omer Levy, Luke Zettlemoyer, and Daniel Weld. 2019. BERT for coreference resolution: Baselines and analysis. In Proceedings of EMNLP-IJCNLP, pages 5807–5812, Hong Kong, China, November.
  • [Kingma and Ba2015] Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of ICLR.
  • [Kundu et al.2018] Gourab Kundu, Avi Sil, Radu Florian, and Wael Hamza. 2018. Neural cross-lingual coreference resolution and its application to entity linking. In Proceedings of ACL, pages 395–400, Melbourne, Australia, July.
  • [Lee et al.2017] Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In Proceedings of EMNLP, pages 188–197, Copenhagen, Denmark, September.
  • [Li and Yarowsky2008] Zhifei Li and David Yarowsky. 2008. Unsupervised translation induction for Chinese abbreviations using monolingual corpora. In Proceedings of ACL, pages 425–433, Columbus, Ohio, June.
  • [Li et al.2019] Ziran Li, Ning Ding, Zhiyuan Liu, Haitao Zheng, and Ying Shen. 2019. Chinese relation extraction with multi-grained information and external linguistic knowledge. In Proceedings of ACL, pages 4377–4386, Florence, Italy, July.
  • [Lin et al.2019] Chen Lin, Timothy Miller, Dmitriy Dligach, Steven Bethard, and Guergana Savova. 2019. A bert-based universal model for both within-and cross-sentence clinical temporal relation extraction. In Proceedings of the 2nd Clinical Natural Language Processing Workshop, pages 65–71.
  • [Liu et al.2009] Hui Liu, Yuquan Chen, and Lei Liu. 2009. Automatic expansion of chinese abbreviations by web mining. In ICAIA, pages 408–416. Springer.
  • [Liu et al.2019a] Xiaodong Liu, Pengcheng He, Weizhu Chen, and Jianfeng Gao. 2019a. Multi-task deep neural networks for natural language understanding. In Proceedings of ACL, pages 4487–4496, Florence, Italy, July.
  • [Liu et al.2019b] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692.
  • [Luong et al.2015] Thang Luong, Hieu Pham, and Christopher D. Manning. 2015.

    Effective approaches to attention-based neural machine translation.

    In Proceedings of EMNLP, pages 1412–1421.
  • [Ma and Hovy2016] Xuezhe Ma and Eduard H. Hovy. 2016. End-to-end sequence labeling via bi-directional lstm-cnns-crf. In Proceedings of ACL.
  • [McCarthy and Lehnert1995] Joseph F. McCarthy and Wendy G. Lehnert. 1995.

    Using decision trees for coreference resolution.

    In Proceedings of IJCAI, pages 1050–1055.
  • [Mintz et al.2009] Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of ACL, pages 1003–1011. Association for Computational Linguistics.
  • [Mitkov1998] Ruslan Mitkov. 1998. Robust pronoun resolution with limited knowledge. In Proceedings of ACL, pages 869–875, Montreal, Quebec, Canada, August.
  • [Muñoz and Montoyo2002] Rafael Muñoz and Andrés Montoyo. 2002. Definite description resolution enrichment with wordnet domain labels. In Francisco J. Garijo, José C. Riquelme, and Miguel Toro, editors, IBERAMIA 2002, pages 645–654, Berlin, Heidelberg. Springer Berlin Heidelberg.
  • [Ng and Cardie2002] Vincent Ng and Claire Cardie. 2002.

    Improving machine learning approaches to coreference resolution.

    In Proceedings of ACL, pages 104–111, Philadelphia, Pennsylvania, USA, July.
  • [Niu et al.2017] Yilin Niu, Ruobing Xie, Zhiyuan Liu, and Maosong Sun. 2017. Improved word representation learning with sememes. In Proceedings of ACL, volume 1, pages 2049–2058.
  • [Ohta et al.2012] Tomoko Ohta, Sampo Pyysalo, Jun’ichi Tsujii, and Sophia Ananiadou. 2012. Open-domain anatomical entity mention detection. In Proceedings of workshop on detecting structure in scholarly discourse, pages 27–36.
  • [Pinillos2011] N Ángel Pinillos. 2011. Coreference and meaning. Philosophical Studies, 154(2):301–324.
  • [Qi et al.2012] Shuhan Qi, Xuan Wang, and Xinxin Li. 2012. Improve coreference resolution with parameter tunable anaphoricity identification and global optimization. In De-Shuang Huang, Yong Gan, Prashan Premaratne, and Kyungsook Han, editors, Bio-Inspired Computing and Applications, pages 307–314, Berlin, Heidelberg. Springer Berlin Heidelberg.
  • [Qi et al.2019] Fanchao Qi, Junjie Huang, Chenghao Yang, Zhiyuan Liu, Xiao Chen, Qun Liu, and Maosong Sun. 2019. Modeling semantic compositionality with sememe knowledge. In Proceedings of ACL, pages 5706–5715, Florence, Italy, July.
  • [Ruslan Mitkov1999] Wolverhampton Wv Sb Ruslan Mitkov. 1999. Anaphora resolution: The state of the art.
  • [Srivastava et al.2014] Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1):1929–1958.
  • [Stenetorp et al.2012] Pontus Stenetorp, Sampo Pyysalo, Goran Topić, Tomoko Ohta, Sophia Ananiadou, and Jun’ichi Tsujii. 2012. Brat: a web-based tool for nlp-assisted text annotation. In Proceedings of EACL, pages 102–107.
  • [Sun et al.2008] Xu Sun, Hou-Feng Wang, and Bo Wang. 2008. Predicting chinese abbreviations from definitions: An empirical learning approach using support vector regression. Journal of Computer Science and Technology, 23(4):602–611.
  • [Tian et al.2019] C. Tian, Y. Zhao, and L. Ren. 2019. A chinese event relation extraction model based on bert. In ICAIBD, pages 271–276, May.
  • [Velickovic et al.2018] Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph attention networks.
  • [Wang et al.2008] Canhui Wang, Zhang Min, Shaoping Ma, and Liyun Ru. 2008. Automatic online news issue construction in web environment. In Proceedings of the 17th International Conference on World Wide Web, WWW 2008, Beijing, China, April 21-25, 2008.
  • [Xu et al.2017] Mingbin Xu, Hui Jiang, and Sedtawut Watcharawittayakul. 2017.

    A local detection approach for named entity recognition and mention detection.

    In Proceedings of ACL, pages 1237–1247, Vancouver, Canada, July.
  • [Xu et al.2019] Hu Xu, Bing Liu, Lei Shu, and Philip Yu. 2019.

    BERT post-training for review reading comprehension and aspect-based sentiment analysis.

    In Proceedings of NAACL, pages 2324–2335, Minneapolis, Minnesota, June.
  • [Zeng et al.2018] Xiangkai Zeng, Cheng Yang, Cunchao Tu, Zhiyuan Liu, and Maosong Sun. 2018. Chinese LIWC lexicon expansion via hierarchical classification of word embeddings with sememe attention. In Proceedings of AAAI, pages 5650–5657.
  • [Zhang and Sun2018] Yi Zhang and Xu Sun. 2018. A chinese dataset with negative full forms for general abbreviation prediction. In Proceedings of LREC.
  • [Zhang and Yang2018] Yue Zhang and Jie Yang. 2018. Chinese NER using lattice LSTM. In Proceedings of ACL, pages 1554–1564.
  • [Zhang et al.2017] Xingxing Zhang, Jianpeng Cheng, and Mirella Lapata. 2017. Dependency parsing as head selection. In Proceedings of EACL, pages 665–676, Valencia, Spain, April.
  • [Zhang et al.2018] Rui Zhang, Cícero Nogueira dos Santos, Michihiro Yasunaga, Bing Xiang, and Dragomir R. Radev. 2018. Neural coreference resolution with deep biaffine attention by joint mention detection and mention clustering. In Proceedings of ACL, pages 102–107.
  • [Zhong1985] Jialing Zhong. 1985. The nature, composition and standardization of abbreviations in modern chinese. Journal of Shenzhen University(Humanities & Social Sciences), (Z1):62–70.
  • [Zhou et al.2019] Wangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei, and Ming Zhou. 2019. BERT-based lexical substitution. In Proceedings of ACL, pages 3368–3373, Florence, Italy, July.