A Pairwise Probe for Understanding BERT Fine-Tuning on Machine Reading Comprehension

06/02/2020 ∙ by Jie Cai, et al. ∙ University of Technology Sydney Peking University 0

Pre-trained models have brought significant improvements to many NLP tasks and have been extensively analyzed. But little is known about the effect of fine-tuning on specific tasks. Intuitively, people may agree that a pre-trained model already learns semantic representations of words (e.g. synonyms are closer to each other) and fine-tuning further improves its capabilities which require more complicated reasoning (e.g. coreference resolution, entity boundary detection, etc). However, how to verify these arguments analytically and quantitatively is a challenging task and there are few works focus on this topic. In this paper, inspired by the observation that most probing tasks involve identifying matched pairs of phrases (e.g. coreference requires matching an entity and a pronoun), we propose a pairwise probe to understand BERT fine-tuning on the machine reading comprehension (MRC) task. Specifically, we identify five phenomena in MRC. According to pairwise probing tasks, we compare the performance of each layer's hidden representation of pre-trained and fine-tuned BERT. The proposed pairwise probe alleviates the problem of distraction from inaccurate model training and makes a robust and quantitative comparison. Our experimental analysis leads to highly confident conclusions: (1) Fine-tuning has little effect on the fundamental and low-level information and general semantic tasks. (2) For specific abilities required for downstream tasks, fine-tuned BERT is better than pre-trained BERT and such gaps are obvious after the fifth layer.



There are no comments yet.


page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1. Introduction

In recent years large pre-trained models (e.g. BERT (Devlin and et al., 2019), RoBERTa (Liu and et al., 2019), XLNet (Yang and et al., 2019), ALBERT (Lan and et al., 2019)

) have brought remarkable advancement to many natural language processing tasks, such as question answering

(Qu and et al., 2019), machine translation (Imamura and Sumita, 2019), natural language inference (Yang et al., 2019)

, name entity recognition

(Sun and Yang, 2019), coreference resolution (Kantor and Globerson, 2019), etc.

Since such large neural models are difficult to explain, many works have analyzed how and why they achieved substantial performances. Lin et al. (2019) and Tenney and et al. (2019) analyze pre-trained models by designing some probing tasks that contain specific linguistic information. Jawahar and et al. (2019) visualizing the clustering method based on the representation of each layer in pre-trained BERT. Clark and et al. (2019) visualize the attention of each head in pre-trained BERT in order to observe the relationship between words. However, few works analyze the impact of fine-tuning, but which is critical to helping us understand its mechanism and thus adapt pre-trained models to complex downstream tasks better. Some existing works (van Aken and et al., 2019; Si and et al., 2019) in this field make qualitative analysis rely on additional model training in probing tasks, but accurate quantitative analysis with less dependency is rare.

To fill the gap, we propose a pairwise ranking approach which is inspired by the observation that many probing tasks involve identifying a matched pair of phrases (e.g. coreference requires matching between an entity and a pronoun). Such quantitative comparison provides high confident conclusions in two aspects: (1) It does not involve model training for probing task, removing possible distraction caused by inaccurate model training; (2) The pairwise metric only concerns with relative ranking comparison, leading to more robustness compared to absolute distance comparison.

Question: what kind of political system does Spain have?
Context: The form of government in Spain is a parliamentary
monarchy, that is, a social…
Answer: a parliamentary monarchy
Question: what happens to the rbc in acute hemolytic reaction?
Context: …as it results from rapid destruction of the donor red
blood cells by host antibodies (IgG , IgM)…
Answer: rapid destruction of the donor red blood cells by host
Question: when did locked out of heaven come out?
Context: …“Locked Out of Heaven” is a song … It was released
as the lead single from the album on October 1, 2012
Answer: October 1, 2012
Answer Type & Boundary
Question: when does the movie battle of the sexes come out?
Context: …released in the United States on September 22, 2017,
by Fox Searchlight Picture…
Answer: September 22, 2017
Table 1. Probing tasks for machine reading comprehension with according examples. Underlined spans within the same question-context example are defined as matching pairs.

In this paper, we take the machine reading comprehension (MRC) task as an example and demonstrate that the proposed pairwise ranking approach is a good way to analyze the impact of fine-tuning. The MRC task is defined as follows: Given a question and a context , a MRC system extracts a span from as the answer according to . Intuitively, given a question, the MRC system needs to first locate to relevant sentence and identify the boundary of . To achieve these goals, we identify five phenomena in MRC:

  • Synonyms identification is required to locate a relevant sentence. In order to answer the corresponding question in Table 1, the model needs to know that “the form of government” in context is a synonym of “political system”.

  • Abbreviation identification is another capability required to locate a relevant sentence. For the corresponding example in Table 1, in order to answer the question, the model needs to know that “rbc” in the question semantically matches “red blood cell” in context.

  • Coreference resolution is closely related to the question. We guarantee the pronoun must be relevant to the answer (pronoun appears in the sentence where the answer locate) and the entity referred to by the pronoun appears in both the question and the context. An example is shown in Table 1. “locked out of heaven” is the entity, the pronoun “it” in the sentence that answer locates refers to the entity “locked out of heavens” in the context.

  • Answer type A good MRC system will determine the type of entity based on the words in the question(where, who, when…) to determine the answer and extract it from the related sentence.

  • Boundary detection

    is necessary for answer extraction. A good MRC system will determine the answer by detecting the boundary and then extract it from the located sentence. We analyze the ability of the model by outputting the logits score of each position.

Long answer Short answer
Model P R F1 P R F1
BERT (pre-train) 16.81 28.10 21.03 4.16 3.81 4.57
BERT (fine-tune) 56.91 62.30 59.48 46.74 40.25 43.25
Table 2. BERT-base performance on NQ dataset.

In the above five phenomena, Abbreviation and Synonyms are two kinds of general capabilities, not unique to the MRC tasks. In other words, these two capabilities can be learned by some common tasks. As for Coreference, Answer type and Boundary, they are unique to span extraction MRC task which means these three capabilities are essential for doing the span extraction MRC task.

In order to probe the above phenomena, we design a pairwise ranking metric to quantitatively compare pre-trained and fine-tuned model with in-domain data. The metric is designed to measure whether matching pairs are closer than random un-matching pairs that aim to provide insight about how well related information are encoded111Our source code is available at https://github.com/hitnq/analyse_bert.. This paper makes the following contributions:

  • We propose a pairwise probe to understand BERT fine-tuning on the MRC task. The probe involves five phenomena in the MRC task and a pairwise ranking metric to quantitatively analyze impact of fine-tuning BERT on the MRC task.

  • We prove that the information learned by BERT during fine-tuning is closely related to the MRC task. Some general linguistic abilities that pre-trained model has not been improved after fine-tuning.

Figure 1.

Comparison of pairwise ranking percentage of pre-trained and fine-tuned BERT model on five types of probing tasks. X-axis represents different layers of BERT, y-axix represents pairwise ranking percentage. Top layer lists results with cosine similarity, and bottom layer lists results with Euclidean distance.

2. Methods

We compare pre-trained BERT and the fine-tuned BERT on a large MRC dataset - Natural Questions (NQ) (Kwiatkowski and et al., 2019). The results that shown in Table 2 indicate fine-tuned BERT outperforms pre-trained BERT by a significant margin on the task. In order to investigate what BERT learned during fine-tuning, we conduct comprehensive case studies and summarize five phenomena in the MRC task. We derive according to test data for each skill and leverage a pairwise ranking metric to compare pre-trained and fine-tuned models.

2.1. Datasets

The training set of NQ has 307,373 examples, the dev set has 7,830 examples and the test set has 7,842 examples. Each example in NQ contains a question and a corresponding Wikipedia page. All questions in NQ presented are real queries issued to Google’s search engine. Answers in each example are annotated with a paragraph (long answer) that answers the corresponding query in the Wikipedia page, while one or more short spans in the paragraph contain the actual answer (short answer). The short answer could be empty. If so, the annotated paragraph is the answer. And if both the long and short answers are empty, then there is no answer on this page. In addition, about 1% of questions can be answered with “yes” or “no”, so the short answer in the example is marked as “yes” or “no”.

2.2. Probing Data

Our pairwise ranking tasks start by constructing challenge datasets from NQ. Each of our tasks consists of examples that include a matching pair and some un-matching pairs.

Synonyms. A word in query and another word in paragraph (the paragraph that answer is located) are a synonymous pair in WordNet222https://wordnet.princeton.edu/. Abbreviation. We ensure that one word in the query is an abbreviation of consecutive phrases in context. Coreference. Our goal is to find a coreference pair that related to the question. First, we identify the entity that exists in the question and paragraph (ensure that the entity does not appear in the sentence where the answer is). Then we find the pronoun in the sentence where the answer locate. The entity and pronoun are taken as the matching pair (see Table 1). Answer Type. We extract examples that question starts with “who”, “when” and “where” because these questions has specific answer types. Boundary. It contains 10k train data and 5k test data from NQ. We simplify the entire paragraph to the sentence where the answer locate.

When build un-matching pairs, we keep question words in the matching pair unchanged and treat each word in the paragraph that does not appear in the matching pair as another word in the un-matching pair. Suppose the number of words in a paragraph is and the number of words in the matching pair of the paragraph is . For a matching pair, we construct para_len-n un-matching pairs.

2.3. Pairwise Ranking Metric

The model we aim to analyze is BERT-base333https://github.com/huggingface/transformers which consist of typically 12 layers of identically sized Transformer(Vaswani and et al., 2017) and embedding size is 768. We can get independent pair representations and in each layer (). We take the average of these tokens’ representations if a word is separated to subtokens by BERT’s tokenizer. Then we calculate the similarity (cosine similarity and Euclidean distance) of a matching pair (pos_sim) and compare it with the similarity of un-matching pairs (neg_sims) to show whether the layer has the ability to distinguish matching pair and un-matching pair. For each example, we suppose the number of words in corresponding paragraph is and .


At last, we calculate the percentage that the similarity of matching pair is higher than un-matching pair in all examples. represents the number of examples and .


3. Results

Figure 1 compares pre-trained and fine-tuned model in terms of pairwise ranking percentage. We have the following observations and analysis.

In both pre-trained and fine-tuned models, almost all representations in high layers show superior capability than lower ones. This phenomenon holds in the cosine similarity evaluation. We think that the different trends in Euclidean distance evaluation are due to differences in evaluation methods.

In both methods, the first five layers of pre-trained and fine-tuned model perform almost the same in all probing tasks. This leads us to conclude that fine-tuning has little effect on the information contained in the first five layers and these layers capture more fundamental and low-level information used in fine-tuned models.

Synonyms & Abbrevation. Fine-tuned model performs similarly to pre-trained model in all layers. The synonyms and abbreviation are more basic abilities that are not closely related to the MRC task. This phenomenon illustrates that pre-trained BERT does indeed contain fundamental linguistic knowledge.

Therefore we conclude that fine-tuning does not affect basic semantic information. The basic semantic information is reflected in the Synonyms and Abbrevation probing tasks and information contained in the first five layers of BERT.

Coreference & Answer Type & Boundary. These three tasks are all necessary for the span extraction MRC task and only when tasks are done can the MRC task be done well. The fine-tuned model has superior performance than pre-trained model in task-specific capabilities and the gap becomes larger after the fifth layer. It means that what the model learns during fine-tuning is related to downstream tasks and the layers after fifth encode task-specific features that are attributed to the gain of scores.

4. Related Work

Recently there are many analytical methods that have emerged mainly focus on analyzing pre-trained and fine-tuned models.

Pre-trained Model Visualization is a intuitive analysis method which we can easily find the relationship between words, and we can clearly see the trend of model changes. For example, Jawahar and et al. (2019) cluster nouns and phrases based on the representation of each layer in pre-trained BERT and demonstrate that pre-trained BERT captures linguistic and semantic information by a series of probing tasks. Clark and et al. (2019) visualize the attention of each head in pre-trained BERT to observe the relationship between two words. Another analysis method is to design some probing tasks to check whether each layer of BERT contains relevant knowledge. Probing tasks are usually designed to contain specific information. For example, POS-tagging is a task that contains surface level information and relation classification is a semantic task. A small classification network is added to the embedding of each layer of BERT. This structure is simple so that it won’t learn too much knowledge to mainly uses the knowledge that Transformer layer has coded (Conneau and et al., 2018; Tenney et al., 2019). Tenney and et al. (2019) introduced a novel “edge probing” method to probe the sub-sentential structure of contextualized word embeddings and applied to nine tasks that encompass a range of syntactic and semantic phenomena.

Fine-tuned Model Some works attempt to analyze what the model learns during fine-tuning. van Aken and et al. (2019) adopted some classification and semantic probing tasks, such as NER, relationship classification, etc. They analyzed the performance of different layers after using MRC datasets fine-tune. The authors also used some clustering methods to visualize the closeness between words in specific task. Kovaleva et al. (2019) summarize five patterns of attention in BERT after fine-tuned by GLUE tasks and and compare the differences between these patterns in pre-trained and fine-tuned versions. There are also some interesting works in adversarial attacks. It verify the robustness of the model through examples that created with specific interference information. Rychalska and et al. (2018b)

perturbed questions while holding the context unchanged and inspect coefficients estimated for ground truth class. The authors also explored the importance of each word in a question by gradually deleting words in the question.

Rychalska and et al. (2018a) verified the importance of verbs in questions by replacing them with their antonyms using WordNet. Si and et al. (2019) proposed two novel constructing different attack data method to verify the robustness of the model that fine-tuned by Multiple-Choice Reading Comprehension (MCRC) tasks. Peters et al. (2019)

compared the performance of “feature extraction” and “fine-tuning” on sentence classification and NLI tasks in GLUE and also compared the effects of different model structures on “feature extraction” and “fine-tuning”.

5. Conclusion

In this paper, we propose a pairwise ranking metric to quantitatively analyze impact of fine-tuning MRC task on BERT, which is closely coupled with the five phenomena in the task. Equipped with such tool, we have verified that pre-trained BERT already gains fundamental and low-level information. In the Synonyms and Abbrevation probing tasks, there is almost no difference after fine-tuning. Our most surprising finding is that, for Coreference, Question Type and Boundary probing tasks, fine-tuning further improves pre-trained model’s capability of conducting task-specific tasks after fifth layer. Moreover, fine-tuning has little effect on the information contained before the fifth layer which verify that fine-tuning does not affect fundamental and low-level information.

In the future, we will apply the pairwise ranking metric to analyze impact of fine-tuning on other tasks. Hopefully such analysis will shed light on more diverse analysis of BERT and thus leading to better understanding of deep learning models.

6. Acknowledgements

This paper was supported by National Key Research and Development Program of China (Grant No. 2017YFB1402400), Ministry of Education ”Tiancheng Huizhi” Innovation Promotes Education Fund (Grant No. 2018B01004)


  • (1)
  • Clark and et al. (2019) Kevin Clark and et al. 2019. What Does BERT Look At? An Analysis of BERT’s Attention. In ACL.
  • Conneau and et al. (2018) Alexis Conneau and et al. 2018. What you can cram into a single

    &!#* vector: Probing sentence embeddings for linguistic properties. In

  • Devlin and et al. (2019) Jacob Devlin and et al. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL.
  • Imamura and Sumita (2019) Kenji Imamura and Eiichiro Sumita. 2019.

    Recycling a Pre-trained BERT Encoder for Neural Machine Translation. In

    Proceedings of the 3rd Workshop on Neural Generation and Translation.
  • Jawahar and et al. (2019) Ganesh Jawahar and et al. 2019. What does BERT learn about the structure of language?. In ACL.
  • Kantor and Globerson (2019) Ben Kantor and Amir Globerson. 2019. Coreference resolution with entity equalization. In ACL.
  • Kovaleva et al. (2019) Olga Kovaleva, Alexey Romanov, Anna Rogers, and Anna Rumshisky. 2019. Revealing the dark secrets of bert. EMNLP (2019).
  • Kwiatkowski and et al. (2019) Tom Kwiatkowski and et al. 2019. Natural questions: a benchmark for question answering research. In ACL.
  • Lan and et al. (2019) Zhen-Zhong Lan and et al. 2019.

    ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In

  • Lin et al. (2019) Yongjie Lin, Yi Chern Tan, and Robert Frank. 2019. Open Sesame: Getting inside BERT’s Linguistic Knowledge. In ACL.
  • Liu and et al. (2019) Yinhan Liu and et al. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. ArXiv abs/1907.11692 (2019).
  • Peters et al. (2019) Matthew E. Peters, Sebastian Ruder, and Noah A. Smith. 2019. To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks. ACL (2019).
  • Qu and et al. (2019) Chen Qu and et al. 2019. BERT with History Answer Embedding for Conversational Question Answering. In SIGIR.
  • Rychalska and et al. (2018a) Barbara Rychalska and et al. 2018a. Does it care what you asked? Understanding Importance of Verbs in Deep Learning QA System. EMNLP (2018).
  • Rychalska and et al. (2018b) Barbara Rychalska and et al. 2018b. How much should you ask? On the question structure in QA systems. EMNLP (2018).
  • Si and et al. (2019) Chenglei Si and et al. 2019. What does BERT Learn from Multiple-Choice Reading Comprehension Datasets? arXiv:cs.CL/1910.12391
  • Sun and Yang (2019) Cong Sun and Zhihao Yang. 2019. Transfer Learning in Biomedical Named Entity Recognition: An Evaluation of BERT in the PharmaCoNER task. In SIGIR.
  • Tenney et al. (2019) Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019. BERT Rediscovers the Classical NLP Pipeline. In ACL.
  • Tenney and et al. (2019) Ian Tenney and et al. 2019. What do you learn from context? Probing for sentence structure in contextualized word representations. In ICLR.
  • van Aken and et al. (2019) Betty van Aken and et al. 2019. How Does BERT Answer Questions?: A Layer-Wise Analysis of Transformer Representations. In CIKM.
  • Vaswani and et al. (2017) Ashish Vaswani and et al. 2017. Attention is All you Need. In NIPS.
  • Yang et al. (2019) Xiaoyu Yang, Xiaodan Zhu, Huasha Zhao, Qiong Zhang, and Yufei Feng. 2019. Enhancing Unsupervised Pretraining with External Knowledge for Natural Language Inference. In

    Canadian Conference on Artificial Intelligence

    . Springer.
  • Yang and et al. (2019) Zhilin Yang and et al. 2019. XLNet: Generalized Autoregressive Pretraining for Language Understanding. In NLPS.