Computational approaches to pragmatics cast language generation and interpretation as game-theoretic or Bayesian inference proceduresGolland et al. (2010); Frank and Goodman (2012)
. While such approaches are capable of modeling a variety of pragmatic phenomena, their main application in natural language processing has been to improve theinformativeness of generated text in grounded language learning problems Monroe et al. (2018). In this paper, we show that pragmatic reasoning can be similarly used to improve performance in more traditional language generation tasks like generation from structured meaning representations (Figure 1) and summarization.
Our work builds on a line of learned Rational Speech Acts (RSA) models Monroe and Potts (2015); Andreas and Klein (2016), in which generated strings are selected to optimize the behavior of an embedded listener model. The canonical presentation of the RSA framework Frank and Goodman (2012) is grounded in reference resolution: models of speakers attempt to describe referents in the presence of distractors, and models of listeners attempt to resolve descriptors to referents. Recent work has extended these models to more complex groundings, including images Mao et al. (2015) and trajectories Fried et al. (2018). The techniques used in these settings are similar, and the primary intuition of the RSA framework is preserved: from the speaker’s perspective, a good description is one that picks out, as discriminatively as possible, the content the speaker intends for the listener to identify.
Outside of grounding, cognitive modeling Frank et al. (2009), and targeted analysis of linguistic phenomena Orita et al. (2015), rational speech acts models have seen limited application in the natural language processing literature. In this work we show that they can be extended to a distinct class of language generation problems that use as referents structured descriptions of lingustic content, or other natural language texts. In accordance with the maxim of quantity Grice (1970) or the Q-principle Horn (1984), pragmatic approaches naturally correct underinformativeness problems observed in state-of-the-art language generation systems ( in Figure 1).
We present experiments on two language generation tasks: generation from meaning representations Novikova et al. (2017) and summarization. For each task, we evaluate two models of pragmatics: the reconstructor-based model of Fried et al. (2018) and the distractor-based model of Cohn-Gordon et al. (2018). Both models improve performance on both tasks, increasing rouge scores by 0.2–0.5 points on the CNN/Daily Mail abstractive summarization dataset and bleu scores by 2 points on the End-to-End (E2E) generation dataset, obtaining new state-of-the-art results.
We formulate a conditional generation task as taking an input from a space of possible inputs (e.g., input sentences for abstractive summarization; meaning representations for structured generation) and producing an output as a sequence of tokens . We build our pragmatic approaches on top of learned base speaker models
, which produce a probability distributionover output text for a given input. We focus on two conditional generation tasks where the information in the input context should largely be preserved in the output text, and apply the pragmatic procedures outlined in Sec. 3 to each task. For these models we use systems from past work that are strong, but may still be underinformative relative to human reference outputs (e.g., Figure 1).
Our first task is generation from structured meaning representations (MRs) containing attribute-value pairs Novikova et al. (2017). An example is shown in Figure 1, where systems must generate a description of the restaurant with the specified attributes. We apply pragmatics to encourage output strings from which the input MR can be identified. For our model, we use a publicly-released neural generation system (Puzikov and Gurevych, 2018) that achieves comparable performance to the best published results in Dušek et al. (2018).
Our second task is multi-sentence document summarization. There is a vast amount of past work on summarizationNenkova and McKeown (2011); recent neural models have used large datasets (e.g., Hermann et al. (2015)) to train models in both the extractive Cheng and Lapata (2016); Nallapati et al. (2017) and abstractive Rush et al. (2015); See et al. (2017) settings. Among these works, we build on the recent abstractive neural summarization system of chen2018fast. First, this system uses a sentence-level extractive model rnn-ext to identify a sequence of salient sentences in each source document. Second, the system uses an abstractive model abs to rewrite each into an output , which are then concatenated to produce the final summary. We rely on the fixed rnn-ext model to extract sentences as inputs in our pragmatic procedure, using abs as our model and applying pragmatics to the abstractive step.
3 Pragmatic Models
To produce informative outputs, we consider pragmatic methods that extend the base speaker models, , using listener models, , which produce a distribution over possible inputs given an output. Listener models are used to derive pragmatic speakers, , which produce output that has a high probability of making a listener model identify the correct input. There are a large space of possible choices for designing and deriving ; we follow two lines of past work which we categorize as reconstructor-based and distractor-based. We tailor each of these pragmatic methods to both our two tasks by developing reconstructor models and methods of choosing distractors.
3.1 Reconstructor-Based Pragmatics
Pragmatic approaches in this category Dušek and Jurčíček (2016); Fried et al. (2018) rely on a reconstructor listener model defined independently of the speaker. This listener model produces a distribution over all possible input contexts , given an output description . We use sequence-to-sequence or structured classification models for (described below), and train these models on the same data used to supervise the models.
The listener model and the base speaker model together define a pragmatic speaker, with output score given by:
where is a rationality parameter that controls how much the model optimizes for discriminative outputs (see Monroe et al. (2017) and Fried et al. (2018) for a discussion). We select an output text sequence for a given input by choosing the highest scoring output under Eq. 1 from a set of candidates obtained by beam search in .
for the meaning representation generation task as a multi-task, multi-class classifier, defining a distribution over possible values for each attribute. Each MR attribute has its own prediction layer and attention-based aggregation layer, which conditions on a basic encoding ofshared across all attributes. See Appendix A.1 for architecture details. We then define as the joint probability of predicting all input MR attributes in from .
To construct for summarization, we train an abs model (of the type we use for , Chen and Bansal (2018)) but in reverse, i.e., taking as input a sentence in the summary and producing a sentence in the source document. We train
on the same heuristically-extracted and aligned source document sentences used to trainChen and Bansal (2018).
3.2 Distractor-Based Pragmatics
Pragmatic approaches in this category Frank and Goodman (2012); Andreas and Klein (2016); Vedantam et al. (2017); Cohn-Gordon et al. (2018) derive pragmatic behavior by producing outputs that distinguish the input from an alternate distractor input (or inputs). We construct a distractor for a given input in a task-dependent way.111In tasks such as contrastive captioning or referring expression generation, these distractors are given; for the conditional generation task, we will show that pragmatic behavior can be obtained by constructing or selecting a single distractor that contrasts with the input .
We follow the approach of cohn2018pragmatically, outlined briefly here. The base speakers we build on produce outputs incrementally, where the probability of , the word output at time , is conditioned on the input and the previously generated words: . Since the output is generated incrementally and there is no separate listener model that needs to condition on entire output decisions, the distractor-based approach is able to make pragmatic decisions at each word rather than choosing between entire output candidates (as in the reconstructor approaches).
The listener and pragmatic speaker are derived from the base speaker and a belief distribution maintained at each timestep over the possible inputs :
where is again a rationality parameter, and the initial belief distribution is uniform, i.e., . Eqs. 2 and 4 are normalized over the true input and distractor ; Eq. 3 is normalized over the output vocabulary. We construct an output text sequence for the pragmatic speaker incrementally using beam search to approximately maximize Eq. 3.
A distractor MR is automatically constructed for each input to be the most distinctive possible against the input. We construct this distractor by masking each present input attribute and replacing the value of each non-present attribute with the value that is most frequent for that attribute in the training data. For example, for the input MR in Figure 1, the distractor is Near[Burger King].
For each extracted input sentence , we use the previous extracted sentence from the same document as the distractor input (for the first sentence we do not use a distractor). This is intended to encourage outputs to contain distinctive information against other summaries produced within the same document.
For each of our two conditional generation tasks we evaluate on a standard benchmark dataset, following past work by using automatic evaluation against human-produced reference text. We choose hyperparameters for our models (beam size, and parametersand ) to maximize task metrics on each dataset’s development set; see Appendix A.2 for the settings used.222Our code is publicly available at https://github.com/sIncerass/prag_generation.
4.1 Meaning Representations
We evaluate on the E2E task of generation from meaning representations containing restaurant attributes Novikova et al. (2017). We report the task’s five automatic metrics: bleu Papineni et al. (2002), nist Doddington (2002), meteor Lavie and Agarwal (2007), rouge-l Lin (2004) and cider Vedantam et al. (2015).
Table 1 compares the performance of our base and pragmatic models to the baseline T-Gen system Dušek and Jurčíček (2016) and the best previous result from the 20 primary systems evaluated in the E2E challenge Dušek et al. (2018). The systems obtaining these results encompass a range of approaches: a template system Puzikov and Gurevych (2018), a neural model Zhang et al. (2018)
, models trained with reinforcement learningGong (2018), and systems using ensembling and reranking Juraska et al. (2018). To ensure that the benefit of the reconstructor-based pragmatic approach, which uses two models, is not due solely to a model combination effect, we also compare to an ensemble of two base models ( ). This ensemble uses a weighted combination of scores of two independently-trained models, following Eq. 1 (with weights tuned on the development data).
Both of our pragmatic systems improve over the strong baseline system on all five metrics, with the largest improvements (2.1 bleu, 0.2 nist, 0.8 meteor, 1.5 rouge-l, and 0.1 cider) from the model. This model outperforms the previous best results obtained by any system in the E2E challenge on bleu, nist, and cider, with comparable performance on meteor and rouge-l.
4.2 Abstractive Summarization
We evaluate on the CNN/Daily Mail summarization dataset Hermann et al. (2015); Nallapati et al. (2016), using See et al.’s (2017) non-anonymized preprocessing. As in previous work Chen and Bansal (2018), we evaluate using rouge and meteor.
Table 2 compares our pragmatic systems to the base model (with scores taken from Chen and Bansal (2018); we obtained comparable performance in our reproduction333We use retrained versions of Chen and Bansal (2018)’s sentence extractor and abstractive models in all our experiments, as well as their n-gram reranking-based inference procedure, replacing scores from the base model
models in all our experiments, as well as their n-gram reranking-based inference procedure, replacing scores from the base modelwith scores from or in the respective pragmatic procedures.), an ensemble of two of these base models, and the best previous abstractive summarization result for each metric on this dataset Celikyilmaz et al. (2018); Paulus et al. (2018); Chen and Bansal (2018). We also report two extractive baselines: Lead-3, which uses the first three sentences of the document as the summary See et al. (2017), and Inputs, the concatenation of the extracted sentences used as inputs to our models (i.e., ).
The pragmatic methods obtain improvements of 0.2–0.5 in rouge scores and 0.2–1.8 meteor over the base model, with the distractor-based approach outperforming the reconstructor-based approach . is strong across all metrics, obtaining results competitive to the best previous abstractive systems.
Coverage ratios for the E2E task by attribute type, estimating how frequently the values for each attribute from the input meaning representations are mentioned in the output text.
The base speaker model is often underinformative, e.g., for the E2E task failing to mention certain attributes of a MR, even though almost all the training examples incorporate all of them. To better understand the performance improvements from the pragmatic models for E2E, we compute a coverage ratio as a proxy measure of how well content in the input is preserved in the generated outputs. The coverage ratio for each attribute is the fraction of times there is an exact match between the text in the generated output and the attribute’s value in the source MR (for instances where the attribute is specified).444Note that this measure roughly provides a lower bound on the model’s actual informativeness for each attribute, since the measure does not assign credit for paraphrases.
Figure 2(a) shows coverage ratio by attribute category for all models. The model increases the coverage ratio when compared to across all attributes, showing that using the reconstruction model score to select outputs does lead to an increase in mentions for each attribute. Coverage ratios increase for as well in four out of six categories, but the increase is typically less than that produced by .
While optimizes less explicitly for attribute mentions than , it still provides a potential method to control generated outputs by choosing alternate distractors. Figure 2(b) shows coverage ratios for when masking only a single attribute in the distractor. The highest coverage ratio for each attribute is usually obtained when masking that attribute in the distractor MR (entries on the main diagonal, underlined), in particular for FamilyFriendly (FF), Food, PriceRange (PR), and Area. However, masking a single attribute sometimes results in decreasing the coverage ratio, and we also observe substantial increases from masking other attributes: e.g., masking either FamilyFriendly or CustomerRating (CR) produces an equal increase in coverage ratio for the CustomerRating attribute. This may reflect underlying correlations in the training data, as these two attributes have a small number of possible values (3 and 7, respectively).
Our results show that models from previous work, while strong, still imperfectly capture the behavior that people exhibit when generating text; and an explicit pragmatic modeling procedure can improve results. Both pragmatic methods evaluated in this paper encourage prediction of outputs that can be used to identify their inputs, either by reconstructing inputs in their entirety or distinguishing true inputs from distractors, so it is perhaps unsurprising that both methods produce similar improvements in performance. Future work might allow finer-grained modeling of the tradeoff between under- and over-informativity within the sequence generation pipeline (e.g., with a learned communication cost model) or explore applications of pragmatics for content selection earlier in the generation pipeline.
Thanks to Reuben Cohn-Gordon for many helpful discussions and suggestions. This work was supported by DARPA through the XAI program. DF is supported by a Tencent AI Lab Fellowship.
- Andreas and Klein (2016) Jacob Andreas and Dan Klein. 2016. Reasoning about pragmatics with neural listeners and speakers. In Proceedings of the Conference on Empirical Methods in Natural Language Processing.
- Bahdanau et al. (2014) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.
- Celikyilmaz et al. (2018) Asli Celikyilmaz, Antoine Bosselut, Xiaodong He, and Yejin Choi. 2018. Deep communicating agents for abstractive summarization. In Proceedings of the Annual Meeting of the North American Chapter of the Association for Computational Linguistics, pages 1662–1675, New Orleans, Louisiana. Association for Computational Linguistics.
- Chen and Bansal (2018) Yen-Chun Chen and Mohit Bansal. 2018. Fast abstractive summarization with reinforce-selected sentence rewriting. In Proceedings of the Annual Meeting of the Association for Computational Linguistics.
- Cheng and Lapata (2016) Jianpeng Cheng and Mirella Lapata. 2016. Neural summarization by extracting sentences and words. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 484–494, Berlin, Germany. Association for Computational Linguistics.
- Cho et al. (2014) Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 1724–1734, Doha, Qatar. Association for Computational Linguistics.
- Cohn-Gordon et al. (2018) Reuben Cohn-Gordon, Noah Goodman, and Chris Potts. 2018. Pragmatically informative image captioning with character-level reference. In Proceedings of the Annual Meeting of the North American Chapter of the Association for Computational Linguistics.
- Doddington (2002) George Doddington. 2002. Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. In Proceedings of the second international conference on Human Language Technology Research, pages 138–145. Morgan Kaufmann Publishers Inc.
- Dušek and Jurčíček (2016) Ondrej Dušek and Filip Jurčíček. 2016. Sequence-to-sequence generation for spoken dialogue via deep syntax trees and strings. In ACL.
- Dušek et al. (2018) Ondřej Dušek, Jekaterina Novikova, and Verena Rieser. 2018. Findings of the E2E NLG challenge. In Proceedings of the International Conference on Natural Language Generation.
- Frank and Goodman (2012) Michael C Frank and Noah D Goodman. 2012. Predicting pragmatic reasoning in language games. Science, 336(6084):998–998.
- Frank et al. (2009) Michael C Frank, Noah D Goodman, Peter Lai, and Joshua B Tenenbaum. 2009. Informative communication in word production and word learning. In Proceedings of the 31st annual conference of the cognitive science society, pages 1228–1233.
- Fried et al. (2018) Daniel Fried, Jacob Andreas, and Dan Klein. 2018. Unified pragmatic models for generating and following instructions. In Proceedings of the Annual Meeting of the North American Chapter of the Association for Computational Linguistics.
- Golland et al. (2010) Dave Golland, Percy Liang, and Dan Klein. 2010. A game-theoretic approach to generating spatial descriptions. In Proceedings of the 2010 conference on Empirical Methods in Natural Language Processing, pages 410–419. Association for Computational Linguistics.
- Gong (2018) Heng Gong. 2018. Technical report for E2E NLG challenge. In E2E NLG Challenge System Descriptions.
- Grice (1970) Herbert P Grice. 1970. Logic and conversation.
- Hermann et al. (2015) Karl Moritz Hermann, Tomás Kociský, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In Advances in Neural Information Processing Systems.
- Horn (1984) Laurence Horn. 1984. Toward a new taxonomy for pragmatic inference: Q-based and r-based implicature. Meaning, form, and use in context: Linguistic applications, 11:42.
- Juraska et al. (2018) Juraj Juraska, Panagiotis Karagiannis, Kevin Bowden, and Marilyn Walker. 2018. A deep ensemble model with slot alignment for sequence-to-sequence natural language generation. In Proceedings of the Annual Meeting of the North American Chapter of the Association for Computational Linguistics, pages 152–162, New Orleans, Louisiana. Association for Computational Linguistics.
- Lavie and Agarwal (2007) Alon Lavie and Abhaya Agarwal. 2007. METEOR: An automatic metric for MT evaluation with high levels of correlation with human judgments. In Proceedings of the Second Workshop on Statistical Machine Translation, pages 228–231, Prague, Czech Republic. Association for Computational Linguistics.
- Lin (2004) Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text Summarization Branches Out: Proceedings of the ACL-04 Workshop.
- Mao et al. (2015) Junhua Mao, Jonathan Huang, Alexander Toshev, Oana Camburu, Alan Yuille, and Kevin Murphy. 2015. Generation and comprehension of unambiguous object descriptions. arXiv preprint arXiv:1511.02283.
- Monroe et al. (2017) Will Monroe, Robert XD Hawkins, Noah D Goodman, and Christopher Potts. 2017. Colors in context: A pragmatic neural model for grounded language understanding. Transactions of the Association for Computational Linguistics.
- Monroe et al. (2018) Will Monroe, Jennifer Hu, Andrew Jong, and Christopher Potts. 2018. Generating bilingual pragmatic color references. In Proceedings of the Annual Meeting of the North American Chapter of the Association for Computational Linguistics.
- Monroe and Potts (2015) Will Monroe and Christopher Potts. 2015. Learning in the Rational Speech Acts model. In Proceedings of 20th Amsterdam Colloquium, Amsterdam. ILLC.
Nallapati et al. (2017)
Ramesh Nallapati, Feifei Zhai, and Bowen Zhou. 2017.
Summarunner: A recurrent neural network based sequence model for extractive summarization of documents.In Proceedings of the Meeting of the Association for the Advancement of Artificial Intelligence.
- Nallapati et al. (2016) Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Caglar Gulcehre, and Bing Xiang. 2016. Abstractive text summarization using sequence-to-sequence rnns and beyond. In SIGNLL Conference on Computational Natural Language Learning, pages 280–290, Berlin, Germany. Association for Computational Linguistics.
- Nenkova and McKeown (2011) Ani Nenkova and Kathleen McKeown. 2011. Automatic summarization. Foundations and Trends in Information Retrieval, 5(2–3):103–233.
- Novikova et al. (2017) Jekaterina Novikova, Ondřej Dušek, and Verena Rieser. 2017. The E2E dataset: New challenges for end-to-end generation. In Proceedings of the Annual SIGdial Meeting on Discourse and Dialogue.
- Orita et al. (2015) Naho Orita, Eliana Vornov, Naomi Feldman, and Hal Daumé III. 2015. Why discourse affects speakers’ choice of referring expressions. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), volume 1, pages 1639–1649.
- Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 311–318. Association for Computational Linguistics.
- Paulus et al. (2018) Romain Paulus, Caiming Xiong, and Richard Socher. 2018. A deep reinforced model for abstractive summarization. In Proceedings of the International Conference on Learning Representations, volume abs/1705.04304.
- Puzikov and Gurevych (2018) Yevgeniy Puzikov and Iryna Gurevych. 2018. E2E NLG challenge: Neural models vs. templates. In Proceedings of the International Conference on Natural Language Generation.
- Rush et al. (2015) Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sentence summarization. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 379–389, Lisbon, Portugal. Association for Computational Linguistics.
- See et al. (2017) Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 1073–1083, Vancouver, Canada. Association for Computational Linguistics.
- Vedantam et al. (2017) Ramakrishna Vedantam, Samy Bengio, Kevin Murphy, Devi Parikh, and Gal Chechik. 2017. Context-aware captions from context-agnostic supervision.
- Vedantam et al. (2015) Ramakrishna Vedantam, C. Lawrence Zitnick, and Devi Parikh. 2015. CIDEr: Consensus-based image description evaluation. In
- Zhang et al. (2018) Biao Zhang, Jing Yang, Qian Lin, and Jinsong Su. 2018. Attention regularized sequence-to-sequence learning for E2E NLG challenge. In E2E NLG Challenge System Descriptions.
Appendix A Supplemental Material
a.1 Reconstructor Model Details
For the reconstructor-based speaker in the E2E task, we first follow the same data preprocessing steps as Puzikov and Gurevych (2018), which includes a delexicalization module that deals with sparsely occurring MR attributes (name, near) by mapping such values to placeholder tokens.
MRs have only a few possible values for most attributes: six out of eight attributes have fewer than seven unique values, and the remaining two attributes (name, near) are handled by our and using delexicalized placeholders, following Puzikov and Gurevych (2018)
. In this way, the reconstructor only needs to predict the presence of these two attributes with a boolean variable, and other attributes with the corresponding categorical variable. We use a one layer bi-directional GRUCho et al. (2014)
for the shared sentence encoder. We concatenate the latent vectors from both directions to construct a bi-directional encoded vectorfor every single word vector as:
Since not all words contribute equally to predicting each MR attribute, we thus use an attention mechanism Bahdanau et al. (2014) to determine the importance of every single word. The aggregated sentence vector for task is calculated by
The task-specific sentence representation is then used as input to layers with softmax outputs, returning a probability vector for each of the MR attributes.
For structured generation, we use beam size 10, , and , tuned to maximize the normalized average of all five metrics on the development set.
For abstractive summarization, we use beam size 20, , and , tuned to maximize rouge-L on the development set.