Conversational Question Reformulation via Sequence-to-Sequence Architectures and Pretrained Language Models

04/04/2020 ∙ by Sheng-Chieh Lin, et al. ∙ 0

This paper presents an empirical study of conversational question reformulation (CQR) with sequence-to-sequence architectures and pretrained language models (PLMs). We leverage PLMs to address the strong token-to-token independence assumption made in the common objective, maximum likelihood estimation, for the CQR task. In CQR benchmarks of task-oriented dialogue systems, we evaluate fine-tuned PLMs on the recently-introduced CANARD dataset as an in-domain task and validate the models using data from the TREC 2019 CAsT Track as an out-domain task. Examining a variety of architectures with different numbers of parameters, we demonstrate that the recent text-to-text transfer transformer (T5) achieves the best results both on CANARD and CAsT with fewer parameters, compared to similar transformer architectures.



There are no comments yet.


page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Natural-language dialogue capabilities play an essential role as an enabling technology in intelligent personal assistants to understand and connect people Gao et al. (2018)

. Effective dialogue systems require many components, including natural language understanding, dialogue state tracking, and natural language generation 

Zhao and Eskenazi (2016). Of late, practitioners in industry Ren et al. (2018) and researchers in academia Elgohary et al. (2019) have made substantial progress in a variety of methods to improve end-to-end task-oriented dialogue systems.

Due to the complex and nuanced nature of human communication, conversations often contain utterances that include coreference, ellipsis, and other phenomena; thus, a good dialogue system should be able to resolve these ambiguities to accurately reconstruct the user’s original intent. We present an example from Elgohary et al. (2019) in Figure 1 to illustrate the task of conversational question reformulation (CQR).

Figure 1: Conversational question reformulation.

However, as we can observe from Figure 1

(a), applying maximum likelihood estimation (MLE) purely based on human-rewritten sentences introduces a strong independence assumption that does not consider conversation dependencies or linguistic structure. Thanks to great progress made by language models pretrained on large corpora using self-supervised learning objectives 

Devlin et al. (2018); Radford et al. (2018); Dong et al. (2019); Raffel et al. (2019), there are now many models equipped with knowledge of various language structures extracted from human-generated texts. We can leverage these models to relax the independence assumption in a pure MLE objective, shown in Figure 1(b).

We list the contributions of this work as follows:

  • [leftmargin=*]

  • We conduct, to our knowledge, the first empirical study leveraging pretrained language models to relax the independence assumption made in using an MLE objective in a CQR task.

  • We achieve the state of the art in terms of BLEU on two CQR benchmarks of task-oriented dialogue systems: (a) conversational open-domain question answering with CANARD and (b) conversational search with TREC CAsT.

In summary, this work demonstrates a simple yet effective way to resolve coreference and ellipsis in a CQR task by leveraging pretrained language models. Furthermore, among representative models, we find that a well-tuned text-to-text transfer transformer (T5) reaches performance that is on par with humans on the in-domain CANARD dataset and achieves the best performance on the out-of-domain CAsT dataset.

2 Related Work

Conversational search Radlinski and Craswell (2017) covers a broad range of techniques that facilitate an IR task in a conversational context: natural language interactions, cumulative clarification Aliannejadi et al. (2019), feedback collection, and information needs profiling during conversations. CQR is an important component of conversational search systems. In order to resolve users’ information needs to retrieve relevant answers, a CQR module that leverages pretrained models is a promising approach, compared to alternatives that track dialogue states based on “cheap” but noisy implicit feedback from users Ahmad et al. (2018, 2019) or “expensive” but sparse judgments Jeffrey et al. (2019).

Open-domain question answering (QA) systems return answers in response to user questions, both in natural language, from a broad range of domains Sun et al. (2018). With great progress coming from contributions by the NLP and IR communities, high quality datasets for single-turn Rajpurkar et al. (2018); Kočiský et al. (2018); Dhingra et al. (2017) and multi-turn (conversational) Reddy et al. (2019); Choi et al. (2018) open-domain QA are available today. These datasets have led to many successful supervised techniques for various tasks Chen et al. (2017); Seo et al. (2017); Huang et al. (2019).

Recently, to improve dialogue understanding, researchers have proposed collecting annotations on resolving multi-turn dialogues in the context of question answering tasks Ren et al. (2018); Elgohary et al. (2019). Building on this line of thought, our work addresses the problem of modeling question rewrites in multi-turn dialogues, especially in the context of open-domain conversational QA.

3 Conversational Question Reformulation

3.1 Problem Formulation

We first formally define the conversational question reformulation (CQR) task, which is also called question de-contextualization Elgohary et al. (2019) or conversational question (query) understanding in the context of task-oriented dialogue systems Ren et al. (2018). Consider a topic from a set of topics , given a topic-oriented utterance sequence (i.e., the conversation history):  of utterances, each of which could be a question or an answer at the -th turn. The task is to reformulate the question into that incorporates the context . In other words, we wish to automatically reformulate the input question by infusing information that exists in the context but is missing from the question itself.

Following the definitions of Ren et al. (2018) and Elgohary et al. (2019), we further refine the task scope of reformulating . Given a question with its historical context and a human-rewritten ground truth , our objective is to induce a function , where is comprised of tokens of length from the context comprising the dialogue sequence (current and historical utterances), modeled as a sequence of tokens of length . The tokens ’s can either be drawn from the context or the current input . In the reconstruction of the ground truth, human annotators are asked to maintain the sentence structure of by copying phrases from the original utterances and performing as few edits as possible.

Finally, given probability

conditioned on a parameterized function and the context (current and historical utterances), the overall objective of the task is then defined in terms of finding the parameters by maximum likelihood estimation:


3.2 Sequence-to-Sequence Architectures and Pretrained Language Models

As both the input and the output are posed in natural language, a reasonable choice for the parametric function is a sequence-to-sequence (S2S) model Sutskever et al. (2014); Vaswani et al. (2017). With this design, we can incorporate context-dependent sentence-level structures when generating output tokens, since the model can consider both the previously-generated sequences as well as the context.

To extract information from the conversation flow, a simple approach, proposed by Xiong et al. (2018) and Elgohary et al. (2019), is to concatenate the historical utterances with the current input as the context , and then use a S2S model to infer the output sequence based on it. To optimize parameters in the S2S model, we can adopt a supervised learning approach to train the S2S model to generate the tokens, given the tokens as ground truth output.

However, Eq (1) makes an important assumption: here, we consider each conversation topic and each -th turn independently. Since a topic-oriented conversation is often coherent and smoothly spans several utterances, an approximation of the parameterized function purely based on Eq (1) could be sub-optimal. To relax this assumption, we introduce pretrained language models Devlin et al. (2018); Radford et al. (2018); Raffel et al. (2019) to leverage language structures extracted from large corpora. Specifically, we adopt these models and fine-tune their pretrained weights, as in previous work Radford et al. (2018); Raffel et al. (2019).

4 Experiments

4.1 Dataset

To evaluate the capability of various models in reformulating conversational questions, we conduct experiments on the CANARD dataset Elgohary et al. (2019), an existing large open-domain dataset for CQR (containing over 30k training samples). Each sample in the CANARD dataset includes an original query from the QuAC dataset Choi et al. (2018), its context (historical utterances and their answers), and the corresponding rewritten question by human annotators.

In addition, we also evaluate model performance on the dataset provided by the TREC 2019 Conversational Assistant Track (CAsT).111 CAsT organizers manually rewrote each conversational query in the evaluation set according to its contextual information and previous utterances in the same session. Statistics of the CANARD and CAsT datasets are presented in Table 1.

Train Dev Test
31,538 3,418 5,571 479
Table 1: Statistics of the datasets used in this work.

4.2 Setup

To train and evaluate our sequence-to-sequence (S2S) models, we construct model input largely following Elgohary et al. (2019). Specifically, we concatenate each original question and its context by adding special separator tokens between them. Separator tokens are also added to contextual information to separate historical utterances. The human-rewritten questions serve as the ground truth target sequences. For encoder- or decoder-only models (e.g., GPT-2, BERT, and UniLM), each training input sequence (as described above) is concatenated with its target sequence, and the models are trained to recover the target sequence using standard masking tricks.

We train each model on the CANARD training set and select the checkpoint with the best performance on development set. In addition to comparing model performance on the CANARD test set, we directly use the model trained on CANARD to perform CQR on the CAsT dataset.222Note that for CAsT, only historical questions are included as contextual information. Model performance is computed by the BLEU score between model output and the human-rewritten ground truth. Table 2 shows the settings of the neural models.

Additional model-specific training details are as follows. (a) LSTM: Following the script provided by Elgohary et al. (2019), we train a bidirectional LSTM S2S model with attention; the word embeddings are initialized with GloVE.333 (b) GPT-2 Radford et al. (2018), which can be characterized as a pretrained decoder-only transformer: To focus on rewriting questions, we fine-tune the model (GPT-2 medium) by masking the cross entropy loss at the positions of the contextual tokens. (c) BERT Devlin et al. (2018), which can be characterized as a pretrained encoder-only transformer: Following the S2S fine-tuning procedure proposed in Dong et al. (2019), we fine-tune BERT-large (cased) by randomly masking the tokens with 70% probability in targeted sequences.444 (d) UniLM Dong et al. (2019), where the model architecture is the same as BERT large and pretrained using three types of language-modeling tasks: The method for fine-tuning is the same as BERT. (e) T5 Raffel et al. (2019), an encoder–decoder transformer that maps natural language understanding tasks to text-to-text transformation tasks: We fine-tune the T5-base model with the same settings used in Nogueira and Lin (2019).555

In addition, we list human performance of CQR (denoted as Human), as measured by Elgohary et al. (2019), and the baseline performance using questions without any reformulation (denoted as Raw) for comparison.

# parameters Learning rate Batch size
LSTM 46M 16
GPT-2-medium 345M 32
BERT-large 340M 32
UniLM-large 340M 32
T5-base 220M 256
Table 2: Model settings.

4.3 Results

Our main results in terms of BLEU on CANARD and CAsT are shown in Table 3, using greedy search decoding for inference. In general, all neural S2S models perform better than the original questions (Raw), except for LSTM on CAsT. This indicates that the PLMs (GPT2, BERT, UniLM, and T5) have obtained at least some generalization capability on the CQR task.

Among all neural S2S models, T5 demonstrates a better ability to learn CQR from human-rewritten questions with fewer model parameters. Specifically, in the CANARD test set, T5 beats the other neural S2S models with 58.08 BLEU, which is close to human performance, 59.92. Furthermore, on CAsT, T5 achieves the highest BLEU score (75.07), four points better than the second-best model (71.21). These results demonstrate T5’s superior generalization ability.

In addition, we also perform S2S model inference using beam search and top- random sampling decoding.666Note that beam search (top- random sampling) is equal to greedy search when the beam width (top-) is set to 1 Figure 2 (left side) shows that beam search with larger beam widths further improves BLEU scores in both datasets. T5 with a beam width of 10 achieves a BLEU score that is on par with human performance on the CANARD test set and reaches 76.22 on CAsT.777We did not perform GPT-2 inference with beam search since the original implementation does not support beam search. Figure 2 (right side) illustrates that random sampling with larger top- leads to poor BLEU scores.888For random sampling, we perform model inference with 10 repetitions and average over them. Under this decoding strategy, T5 still maintains better BLEU scores compared to the other S2S models.

Dev Test
Human 59.92 -
Raw 33.84 36.25 60.41
LSTM 43.68 39.15 42.24
GPT-2-medium 52.63 50.07 68.07
BERT-large 55.34 54.34 69.53
UniLM-large 57.39 55.92 71.21
T5-base 59.13 58.08 75.07
Table 3: BLEU score comparison. For simplicity, we compare neural S2S models using greedy search.
Figure 2: Decoding sensitivity analysis

5 Conclusion

In this paper, we conduct experiments on conversational question reformulation (CQR) via neural sequence-to-sequence (S2S) models and demonstrate that our fine-tuned T5-base model achieves the state of the art, in one case achieving performance on par with humans (at least measured by BLEU). In addition, experiments on the CAsT dataset show that our fine-tuned T5-base model can be directly used in a transfer setting and beats other neural S2S models by quite a large margin.

6 Acknowledgements

This research was supported in part by the Canada First Research Excellence Fund and the Natural Sciences and Engineering Research Council (NSERC) of Canada. Additionally, we would like to thank Google for computational resources in the form of Google Cloud credits.


  • Ahmad et al. (2018) Wasi Uddin Ahmad, Kai-Wei Chang, and Hongning Wang. 2018. Multi-task learning for document ranking and query suggestion. In Proc. ICLR.
  • Ahmad et al. (2019) Wasi Uddin Ahmad, Kai-Wei Chang, and Hongning Wang. 2019. Context attentive document ranking and query suggestion. In Proc. SIGIR, page 385–394.
  • Aliannejadi et al. (2019) Mohammad Aliannejadi, Hamed Zamani, Fabio Crestani, and W. Bruce Croft. 2019. Asking clarifying questions in open-domain information-seeking conversations. In Proc. SIGIR, page 475–484.
  • Chen et al. (2017) Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading Wikipedia to answer open-domain questions. In Proc. ACL, pages 1870–1879.
  • Choi et al. (2018) Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, and Luke Zettlemoyer. 2018. QuAC: Question answering in context. arXiv:1808.07036.
  • Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805.
  • Dhingra et al. (2017) Bhuwan Dhingra, Kathryn Mazaitis, and William W. Cohen. 2017. Quasar: Datasets for question answering by search and reading. arXiv:1707.03904.
  • Dong et al. (2019) Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon. 2019. Unified language model pre-training for natural language understanding and generation. In Proc. NIPS.
  • Elgohary et al. (2019) Ahmed Elgohary, Denis Peskov, and Jordan Boyd-Graber. 2019. Can you unpack that? Learning to rewrite questions-in-context. In Proc. EMNLP, pages 5917–5923.
  • Gao et al. (2018) Jianfeng Gao, Michel Galley, and Lihong Li. 2018. Neural approaches to conversational AI. arXiv:1809.08267.
  • Huang et al. (2019) Hsin-Yuan Huang, Eunsol Choi, and Wen tau Yih. 2019. FlowQA: Grasping flow in history for conversational machine comprehension. In Proc. ICLR.
  • Jeffrey et al. (2019) Dalton Jeffrey, Chenyan Xiong, and Jamie Callan. 2019. CAsT 2019: The conversational assistance track overview. In Proc. TREC.
  • Kočiský et al. (2018) Tomáš Kočiský, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette. 2018. The NarrativeQA reading comprehension challenge. Trans.of ACL, 6:317–328.
  • Nogueira and Lin (2019) Rodrigo Nogueira and Jimmy Lin. 2019. From doc2query to docTTTTTquery.
  • Radford et al. (2018) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2018. Language models are unsupervised multitask learners.
  • Radlinski and Craswell (2017) Filip Radlinski and Nick Craswell. 2017. A theoretical framework for conversational search. In Proc. CHIIR, page 117–126.
  • Raffel et al. (2019) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv:1910.10683.
  • Rajpurkar et al. (2018) Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don’t know: Unanswerable questions for SQuAD. In Proc. ACL, pages 784–789.
  • Reddy et al. (2019) Siva Reddy, Danqi Chen, and Christopher D. Manning. 2019. CoQA: A conversational question answering challenge. Trans. of ACL, 7:249–266.
  • Ren et al. (2018) Gary Ren, Xiaochuan Ni, Manish Malik, and Qifa Ke. 2018. Conversational query understanding using sequence to sequence modeling. In Proc. WWW, page 1715–1724.
  • Seo et al. (2017) Min Joon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. Bidirectional attention flow for machine comprehension. In Proc. ICLR.
  • Sun et al. (2018) Haitian Sun, Bhuwan Dhingra, Manzil Zaheer, Kathryn Mazaitis, Ruslan Salakhutdinov, and William Cohen. 2018. Open domain question answering using early fusion of knowledge bases and text. In Proc. EMNLP, pages 4231–4242.
  • Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014.

    Sequence to sequence learning with neural networks.

    In Proc. NIPS, page 3104–3112.
  • Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proc. NIPS, pages 5998–6008.
  • Xiong et al. (2018) Wayne Xiong, Lingfeng Wu, Jun Zhang, and Andreas Stolcke. 2018. Session-level language modeling for conversational speech. In Proc. EMNLP, pages 2764–2768.
  • Zhao and Eskenazi (2016) Tiancheng Zhao and Maxine Eskenazi. 2016.

    Towards end-to-end learning for dialog state tracking and management using deep reinforcement learning.

    In Proc. of SIGDIAL, pages 1–10.