End-to-end neural response generation methods are promising for developing open domain dialogue systems as they allow to learn from very large unlabeled datasets Shang et al. (2015); Sordoni et al. (2015); Vinyals and Le (2015). However, these models have also been shown to generate generic, uninformative, and non-coherent replies (e.g., “I don’t know.” in Figure 1), mainly due to the fact that neural systems tend to settle for the most frequent options, thus penalizing length and favoring high-frequency word sequences Sountsov and Sarawagi (2016); Wei et al. (2017).
To address these problems, JiweiLi:diversity2015 and li2017learning attempt to promote diversity by improving the objective function, but do not model diversity explicitly. Serban2017AHL focus on model structure without any upgrades to the objective function. Other works control the style of the output by leveraging external resources (hu2017toward: sentiment classifier, time annotation; zhao2017learning: dialogue acts) or focus on well-structured input such as paragraphsLi and Jurafsky (2017).
This paper extends previous attempts to model diversity and coherence by enhancing all three aspects of the learning process: the data, the model, and the objective function. While previous research has addressed these aspects individually, this paper is the first to address all three in a unified framework. Instead of using existing linguistic knowledge or labeled datasets, we aim to control for coherence by learning directly from data, using a fully unsupervised approach. This is also the first work encoding and evaluating coherence explicitly in the dialogue generation task, as opposed to using diversity, style, or other properties of responses as a proxy.
|A: You stay out of this.||B-Coh: Well, I got water.|
|B: So you want water, huh?||B-Incoh: I don’t know.|
|A: That’s right.|
|A: Where do we start?||B-Coh: Specifically the stove.|
|B: Kitchen.||B-Incoh: Let’s go for a walk.|
|A: Definitely the kitchen.|
In this work, given a dialogue history, we regard as a coherent response an utterance that is thematically correlated and naturally continuing from the previous turns, as well as lexically diverse. For example, in Figure 1 the response “Specifically the stove.” is a very natural and coherent response, elaborating on the topic of kitchen introduced in the previous two utterances and containing rich thematic words, whereas the response “Let’s go for a walk.” is unrelated and uninteresting.
In order to obtain coherent responses, we present three generic enhancements to existing encoder-decoder (E-D) models:
We filter a corpus of conversations based on our measure of coherence, which leaves us with context-response pairs that are both topically coherent and lexically diverse.
We train an E-D generator recast as a conditional Variational Autoencoder (cVAE; Zhao et al., 2017) model that incorporates two latent variables, one for encoding the context and another for conditioning on the measure of coherence, trained jointly as in hu2017toward. We then decode using a context gate Tu et al. (2017) to control the generation of words that directly relate to the most topical words of the context and promote coherence.
Experiments on the OpenSubtitles Lison and Meena (2016) corpus demonstrate the effectiveness of the overall approach. Our models achieve a substantial improvement over competitive neural models. We provide an ablation analysis, quantifying the contributions that come from effective modeling of coherence into our models. All our experimental code is freely available on GitHub.111https://github.com/XinnuoXu/CVAE_Dial
2 Coherence-based Dialogue Generation
Our model aims to generate responses given a dialogue context, incorporating measures of coherence estimated purely from the training data. We propose the following enhancements to the attention-based E-D architectureBahdanau et al. (2015); Luong et al. (2015):
We force the model to condition on the measure of coherence explicitly by encoding a latent variable (code) learned from data.
We incorporate a context gate Tu et al. (2017) that dynamically controls the ratio at which the generated words in the response derive directly from the coherence-enhanced dialogue context or the previously generated parts of the response.
2.1 Measure of Dialogue Coherence
Semantic vector space models of language represent each word with a real-valued word embedding vector Pennington et al. (2014). By simply taking a weighted average of all its word embeddings, a whole sentence can be mapped into the semantic vector space. We define the coherence of a dialogue as the average distance between semantic vectors of preceding dialogue context and its response.
Let represent a dialogue context and a response. and are the numbers of words in the dialogue context and its response, respectively. Semantic vector space models map each word into embeddings , and into . The semantic representation of a dialogue context is then ; for a response , it is . Here, and are importance weights for each word in the sentence.222We set the importance weights to 0 for a list of stop words (high-frequency words such as articles and prepositions, names, punctuation marks), 1 otherwise. The measure of coherence is then defined as the cosine distance of the two semantic vectors of the dialogue context and its response:
2.2 Model Overview
End-to-end response generation for dialogue can be formalized as follows: Given a dialogue context , a dialogue generator generates the next utterance
. During the training process, the aim for a dialogue generator is to maximize the probabilityover the training dataset. To encode dialogue contexts that adequately incorporate coherence information, we build our generator based on the cVAE model of Hu et al. (2017)
, which has been used to control text generation with respect to linguistic properties, such as tense or sentiment.
In our model, the response is generated conditioned on the previous conversation , a diversity-promoting latent variable , and a latent variable indicating dialogue coherence; and are independent. The generation probability is defined as:
Unfortunately, optimizing Eq (2) during training is intractable; therefore, we apply variational inference and optimize instead the variational lower bound:
where is the probability of generating utterance given , and ; stands for the approximate posterior distribution of the latent variable conditioned on dialogue context and the gold response ; is the measure of coherence between context and response ; is the true prior distribution of conditioned only on dialogue context ; denotes the KL-divergence. We assume that both and are Gaussian with mean vectors , and covariance matrices , .
2.2.1 Model Details
Optimizing Eq (3) consists of two parts: (1) minimizing the KL-divergence between the approximate posterior distribution and the true prior distribution of , (2) maximizing the probability of generating the gold response conditioned on dialogue context and coherence factors and . Figure 2 shows the pipeline of the training procedure.
First, we encode a dialogue context into a hidden state using the context encoder
, which is based on Recurrent Neural Networks (RNNs). Then theposterior network encodes both dialogue context and gold response into a hidden state
followed by two linear transformationsand to map into mean vector and covariance matrix . The latent variable can be sampled from the distribution :
The prior network in Figure 2 takes a form similar to the posterior network:
where is the final hidden state of an RNN encoding only the dialogue context , and , are linear transformations. Code is given by the coherence measure from Eq (1).
We build an attention-based decoder Bahdanau et al. (2015); Luong et al. (2015) using RNNs to generate responses conditioned on encoded dialogue context , diversity signal , and coherence signal . We concatenate the latent variables and to the context encoder hidden state and feed them into the decoder as the initial hidden state , similar to hu2017toward.
During the decoding process, tokens are generated sequentially under the following probability distribution:
where is the length of the produced response; is an RNN; is the hidden state of the decoder at time step which is conditioned on the previously generated token , the previous hidden state , and the weighted attention vector :
where is the number of tokens of the dialogue context; is the hidden state of the encoder; the attention weight of each context hidden state is computed following luong2015effective.
To increase the influence of code , we introduce the context gate . Unlike tu2016context, whose context gate assigns an element-wise weight to the input signal deriving from the encoder RNN, we build the context gate conditioned only on the coherence signal:
is the sigmoid function;is a bias term;333We set empirically against the development set. is the target value of the measure of coherence, calculated by (see Section 2.1); is the measure of coherence between the dialogue context and the generated prefix sentence at time step , calculated by . Now Eq (7) with the context gate applied to can be rewritten as:
where denotes element-wise multiplication.
The coherence-informed context gate aims to dynamically control the ratio at which preceding dialogue context and previously generated tokens of the current response contribute to the generation of the next token in the response.
Our generator is trained similarly to hu2017toward. The objective function is a weighted combination of three losses (generation, coherence, and diversity):
To teach the generator to produce responses close to the training data, we maximize the generation probability of the training response given the dialogue context according to Eq (2). During training, we set and minimize the following:
Apart from the generation loss, the coherence measure provides an extra learning signal which pushes the generator to produce responses that match the coherence signal given by the latent variable .
In Eq (13), is the prior distribution of the coherence variable . To ensure that the loss is differentiable, we cannot sample words from the response vocabulary. Instead we define as the sequence of output word probability distributions. is predicted by the coherence measure defined in Eq (1) with set as:
where is the word embedding matrix trained using GloVe (Section 2.1).
The last component in Eq (11) is the independent constraint that forces the soft distribution over the generated response to be diverse, so that it is able to faithfully reproduce the latent variable :
where is predicted by the posterior network with as the soft input to the RNN encoder at each time step .
Figure 3 shows the inference process of the generative model. Given a dialogue context and an expected coherence value , the context encoder first encodes the dialogue context into a hidden state . The prior network then generates a sample conditioned on the dialogue context. The decoder is initialized with , i.e., the concatenation of , and . During decoding, the next word is generated via the context gate modulating between the attention-reweighted context and the previously generated words of the response.
3 Dataset and Filtering
Dataset for Generator
We train and evaluate our models on the OpenSubtitles corpus Lison and Tiedemann (2016) with automatic dialogue turn segmentation Lison and Meena (2016).444http://www.opensubtitles.org A training pair consists of a dialogue context and a corresponding response. We consider three consecutive turns as the dialogue context and the following turn as the response. From a total of 65M instances, we select those that have context and response lengths of less than 120 and 30 words, respectively. We create two datasets:
OST (plain OpenSubtitles) consists of 2M/4K/4K instances as our training/development/test sets, selected randomly from the whole corpus;
fOST (filtered OpenSubtitles) contains the same amount of instances, but randomly selected only among those that have a measure of coherence score .555The coherence score is calculated as shown in Eq (1
). We observed that the scores on the training set follow a normal distribution with a slight tail on the negatively correlated side, so we fit a normal distribution to the data with parametersand set the cut-off to . A histogram of coherence scores is shown in Figure 5 in Supplemental Material A.
Filtering of the OpenSubtitles corpus is motivated by the fact that by removing the video and audio modalities which the subtitles originally accompanied, we are very often left with incomplete and incoherent dialogues. Therefore, by keeping dialogues with high coherence scores, we aim at building a high quality corpus with (1) more semantically coherent and topically related contexts and responses, and (2) fewer general and dull responses. Table 1 shows the coherence and diversity metrics (cf. Section 4.2) between OST and fOST. Unsurprisingly, coherence for fOST is much higher than OST, with a slightly higher diversity. We list dialogue examples for different coherence scores in Supplemental Material B.
Dataset for Coherence Measure
In order to accurately measure coherence on our domain using the semantic distance as defined in Section 2.1, we train GloVe embeddings on the full OpenSubtitles corpus (i.e. 100K movies).
Our generator model, ablative variants, and baselines are implemented using the publicly available OpenNMT-py framework Klein et al. (2017) based on bahdanau2014neural and luong2015effective. We used the publicly available glove-python package888https://github.com/maciejkula/glove-python to implement our coherence measure.
We experiment on two versions of our model: (1) cVAE with the coherence context gate as described in Section 2.3 (cVAE-XGate), (2) cVAE with the original context gate implementation of Tu et al. (2017) (cVAE-CGate). For each of these, we consider the main variant where the input coherence measure is preset to a fixed ideal value as estimated on development data (1.0 for OST and 0.95 for fOST), as well as an oracle variant where we use the true coherence measure between the context and the gold-standard response in the test set (indicated with “(C)” in Tables 2 and 3).
We compare against two baseline models: (1) a vanilla E-D with attention (Attention) Luong et al. (2015); (2) an enhancement where output beams are rescored using the maximum mutual information anti-language model (MMI-antiLM) of Li et al. (2016a) (MMI).
4.1 Parameter Settings
We set our model parameters based on preliminary experiments on the development data.
We use 2-layer RNNs with LSTM cells Hochreiter and Schmidhuber (1997) with input/hidden dimension of 128 for both the context encoder and the decoder. The dropout rate is set to 0.2 and the Adam optimizer Kingma and Ba (2015) is used to update the parameters. A vocabulary of 25,000 words is shared between the encoder and the decoder.
Both the posterior network and prior network for the latent variable learning are built with 2-layer LSTM RNNs with input/hidden dimension of 64. The dimension of the latent variable is set to 20. Same as for the encoder and decoder, the dropout rate is 0.2 and the Adam optimizer is used to update the parameters.
The window size for GloVe computation in our coherence measure is set to 10.
4.2 Evaluation metrics
We use a number of metrics to evaluate the outputs of our models:
BLEU, B1, B2, B3 – the word-overlap score against gold-standard responses Papineni et al. (2002) used by the vast majority of recent dialogue generation works Zhao et al. (2017); Yao et al. (2017); Li et al. (2017a, 2016c); Sordoni et al. (2015); Li et al. (2016a); Ghazvininejad et al. (2017). BLEU in this paper refers to the default BLEU-4, but we also report on lower -gram scores (B1, B2, B3).999We use the Multi-BLEU script from OpenNMT to measure BLEU scores.
Coh – our novel GloVe-based coherence score calculated using Eq (1) showing the semantic distance of dialogue contexts and generated responses.
All model variants described in Section 4 are trained on both OST and fOST datasets. Tables 2 and 3 present the scores of all models tested on the OST and fOST test sets, respectively. Note that in addition to testing the models on the respective test sections of their training datasets, we also test them on the other dataset (OST-trained models on fOST and vice-versa). This way, we can observe the performance of the fOST-trained models in more noisy contexts and see how good the OST-trained models are when evaluated against coherent responses only.
Given all the evaluated model variants, we can observe the effects and contributions of the individual components of our setup:
The models trained on fOST consistently outperform the same models trained on OST – for all evaluation metrics and on both test sets. This shows that coherence-based training data filtering is generally beneficial.
cVAE-Context Gate models: Nearly all cVAE-based models perform markedly better than the baselines w.r.t. BLEU, coherence, and diversity.101010 We performed paired bootstrap re-sampling for the best cVAE model and the best baseline model in each experiments set (Table 2 and Table 3) as is done for MT (Koehn, 2004), which confirmed statistical significance at 99% confidence level for all cases except for models trained on fOST and tested on OST (bottom half of Table 2).
If we look at models trained on OST and tested on fOST (the top half of Table 3), we can see that all cVAE-based models, especially cVAE-XGate, are able to learn to produce coherent and diverse response even when trained on a noisy, incoherent corpus. Examples of responses generated by the baseline MMI model and by cVAE-XGate in Figure 4 show that cVAE-XGate mostly produces more diverse and coherent responses than MMI.
Preset vs. oracle models with gold-standard : Table 2 shows that on the noisy OST test set, cVAE-based models using the gold-standard value of achieve higher BLEU scores than models using preset . This is expected since many gold-standard responses in the unfiltered set have a low coherence score; – the model can generate a more generic response if the gold-standard is low. The models with preset always attempt to generate coherent responses, which is apparent from the other metrics: Coh and D-Sent are consistently higher than for models using gold-standard .
On the fOST test set where only high-coherence responses are expected, models using fixed consistently reach higher scores in all metrics including BLEU (see Table 3). This shows that in general, using a preset constant value of works well, even better than using the gold-standard .
In sum, using our coherence measure both for data filtering and inside the models leads to output performance improvements.
|A:||i have an audition at 4:00.||B-GT: you volunteered.|
|1||B:||apparently now i’m cooking dinner.||B-MMI: i don ’t know.|
|A:||how did this all happen?||B-XGT: well , i’m going to have a dinner.|
||A:||it’s not that easy.||B-GT: the waiting list would take too long.|
|2||B:||we can’t contact potential donors.||B-MMI: i don’t know.|
|A:||if we want to save frank, his brother is his only hope||B-XGT: we can’t save him.|
||A:||that fish ate my fish!||B-GT: you’ll wait till next year to win one of these.|
|3||B:||you cost me £30,000||B-MMI: i’m sorry.|
|A:||sorry, folks.||B-XGT: you owe me a little fish.|
||A:||was it what you wanted?||B-GT: you like model planes?|
|4||B:||no.||B-MMI: i wanted to know.|
|A:||i wanted an airplane.||B-XGT: i wanted a helicopter.|
||A:||great.||B-GT: yeah, a vanilla decaf latte.|
|5||B:||thanks so much.||B-MMI: no.|
|A:||vanilla decaf latte?||B-XGT: yeah , that ’s a good coffee|
||A:||to make everybody happy.||B-GT: and stick it in your mouth.|
|6||B:||which means that sometimes||B-MMI: and you got to take a shower.|
|A:||you got to take a pile of poop||B-XGT: and you got to take a walk.|
||A:||now you’ll knock again and say you forgot to ask||B-GT: should i knock and make her come here for that?|
|7||B:||if she has a friend for your friend.||B-MMI: you’re crazy!|
|A:||are you crazy?||B-XGT: i’m crazy, i’m crazy.|
||A:||i knew you would say that.||B-GT: you are all way too pretty to be arguing.|
|8||B:||mr. kerem is a charming and decent guy.||B-MMI: he’s a very good man.|
|A:||and he’s really into you.||B-XGT: i didn’t say that.|
6 Related Work
Our work fits into the context of the very active area of end-to-end generative conversation models, where neural E-D approaches have been first applied by Vinyals and Le (2015) and extended by many others since.
Many works address the lack of diversity and coherence in E-D outputs Sountsov and Sarawagi (2016); Wei et al. (2017) but do not attempt to model coherence directly, unlike our work: Li et al. (2016a) use anti-LM reranking; Li et al. (2016c) modify the beam search decoding algorithm, similar to Shao et al. (2017)
in addition to using a self-attention model.Mou et al. (2016) predict keywords for the output in a preprocessing step while Wu et al. (2018) preselect a vocabulary subset to be used for decoding. Li et al. (2016b) focus specifically on personality generation (using personality embeddings) and Wang et al. (2017) promote topic-specific outputs by language-model rescoring and sampling.
A lot of recent works explore the use of additional training signals and VAE setups in dialogue generation. In contrast to this paper, they do not focus explicitly on coherence: Asghar et al. (2017)
use reinforcement learning with human-provided feedback,Li et al. (2017a) use a RL scenario with length as reward signal. Li et al. (2017b) add an adversarial discriminator to provide RL rewards (discriminating between human and machine outputs), Xu et al. (2017) use a full adversarial training setup. The most recent works explore the usage of VAEs: Cao and Clark (2017) explore a vanilla VAE setup conditioned on dual encoder (for contexts and responses) during training, the model of Serban et al. (2017) uses a VAE in a hierarchical E-D model. Shen et al. (2017) use a cVAE conditioned on sentiment and response genericity (based on a handwritten list of phrases). Shen et al. (2018) combine a cVAE with a plain VAE in an adversarial fashion.
We also draw on ideas from other areas than dialogue generation to build our models: tu2016context’s context gates originate from machine translation and hu2017toward’s cVAE training stems from free-text generation.
7 Conclusions and Future Work
We showed that explicitly modeling coherence and optimizing towards coherence and diversity leads to better-quality outputs in dialogue response generation. We introduced three extensions to current encoder-decoder response generation models: (1) we defined a measure of coherence based on GloVe embeddings Pennington et al. (2014), (2) we filtered the OpenSubtitles training corpus Lison and Meena (2016) based on this measure to obtain coherent and diverse training instances, (3) we trained a cVAE model based on Hu et al. (2017) and Tu et al. (2017) that uses our coherence measure as one of the training signals. Our experimental results showed a considerable improvement in the output quality over competitive models, which demonstrates the effectiveness of our approach.
In future work, we plan to replace the GloVe-based measure of coherence with a trained discriminator that distinguishes between coherent and incoherent responses Li and Jurafsky (2017). This will allow us to use extend the notion of coherence to account for phenomena such as topic shifts. We also plan to verify the results with a human evaluation study.
This research received funding from the EPSRC project MaDrIgAL (EP/N017536/1). The Titan Xp used for this research was donated by the NVIDIA Corporation.
- Asghar et al. (2017) Nabiha Asghar, Pascal Poupart, Xin Jiang, and Hang Li. 2017. Deep Active Learning for Dialogue Generation. In 6th Joint Conference on Lexical and Computational Semantics (*SEM) 2017. ArXiv: 1612.03929.
- Bahdanau et al. (2015) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural Machine Translation by Jointly Learning to Align and Translate. In 3rd International Conference on Learning Representations (ICLR2015), San Diego, CA, USA. ArXiv: 1409.0473.
- Bowman et al. (2016) Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew Dai, Rafal Jozefowicz, and Samy Bengio. 2016. Generating Sentences from a Continuous Space. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages 10–21, Berlin, Germany. Association for Computational Linguistics.
- Cao and Clark (2017) Kris Cao and Stephen Clark. 2017. Latent Variable Dialogue Models and their Diversity. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 182–187, Valencia, Spain. Association for Computational Linguistics.
- Chung et al. (2015) Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron C Courville, and Yoshua Bengio. 2015. A recurrent latent variable model for sequential data. In Advances in neural information processing systems, pages 2980–2988.
- Dhingra et al. (2017) Bhuwan Dhingra, Lihong Li, Xiujun Li, Jianfeng Gao, Yun-Nung Chen, Faisal Ahmed, and Li Deng. 2017. Towards end-to-end reinforcement learning of dialogue agents for information access. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 484–495.
- Ghazvininejad et al. (2017) Marjan Ghazvininejad, Chris Brockett, Ming-Wei Chang, Bill Dolan, Jianfeng Gao, Wen tau Yih, and Michel Galley. 2017. A knowledge-grounded neural conversation model. arXiv preprint arXiv:1702.01932.
- Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780.
Hu et al. (2017)
Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P Xing.
Toward controlled generation of text.
International Conference on Machine Learning, pages 1587–1596.
- Kingma and Ba (2015) Diederik Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In Proceedings of the 3rd International Conference on Learning Representations, San Diego, CA, USA. ArXiv: 1412.6980.
Klein et al. (2017)
Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander M. Rush.
OpenNMT: Open-source toolkit for neural machine translation.In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, Canada.
Philipp Koehn. 2004.
Statistical significance tests for machine translation evaluation.
Proceedings of the 2004 conference on empirical methods in natural language processing.
- Li et al. (2016a) Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016a. A diversity-promoting objective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 110–119. Association for Computational Linguistics.
- Li et al. (2016b) Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016b. A Persona-Based Neural Conversation Model. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany. ArXiv: 1603.06155.
- Li and Jurafsky (2017) Jiwei Li and Dan Jurafsky. 2017. Neural Net Models for Open-Domain Discourse Coherence. In Empirical Methods in Natural Language Processing, Copenhagen, Denmark. ArXiv: 1606.01545.
- Li et al. (2016c) Jiwei Li, Will Monroe, and Dan Jurafsky. 2016c. A simple, fast diverse decoding algorithm for neural generation. arXiv preprint arXiv:1611.08562.
- Li et al. (2017a) Jiwei Li, Will Monroe, and Dan Jurafsky. 2017a. Learning to decode for future success. arXiv preprint arXiv:1701.06549.
- Li et al. (2017b) Jiwei Li, Will Monroe, Tianlin Shi, Alan Ritter, and Dan Jurafsky. 2017b. Adversarial Learning for Neural Dialogue Generation. arXiv:1701.06547 [cs]. ArXiv: 1701.06547.
- Lison and Meena (2016) Pierre Lison and Raveesh Meena. 2016. Automatic Turn Segmentation for Movie & TV Subtitles. In 2016 IEEE Workshop on Spoken Language Technology. IEEE conference proceedings.
- Lison and Tiedemann (2016) Pierre Lison and Jörg Tiedemann. 2016. OpenSubtitles2016: Extracting Large Parallel Corpora from Movie and TV Subtitles. In Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC 2016), Portorož, Slovenia.
- Luong et al. (2015) Minh-Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective Approaches to Attention-based Neural Machine Translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1412–1421, Lisbon, Portugal. ArXiv: 1508.04025.
- Mou et al. (2016) Lili Mou, Yiping Song, Rui Yan, Ge Li, Lu Zhang, and Zhi Jin. 2016. Sequence to Backward and Forward Sequences: A Content-Introducing Approach to Generative Short-Text Conversation. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3349–3358, Osaka, Japan. The COLING 2016 Organizing Committee. ArXiv: 1607.00970.
- Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311–318. Association for Computational Linguistics.
- Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar. Association for Computational Linguistics.
- Serban et al. (2017) Iulian Serban, Alessandro Sordoni, Ryan Joseph Lowe, Laurent Charlin, Joelle Pineau, Aaron C. Courville, and Yoshua Bengio. 2017. A hierarchical latent variable encoder-decoder model for generating dialogues. In AAAI.
- Shang et al. (2015) Lifeng Shang, Zhengdong Lu, and Hang Li. 2015. Neural responding machine for short-text conversation. In Proceedings of Annual Meeting of the Association for Computational Linguistics (ACL).
- Shao et al. (2017) Louis Shao, Stephan Gouws, Denny Britz, Anna Goldie, Brian Strope, and Ray Kurzweil. 2017. Generating Long and Diverse Responses with Neural Conversation Models. In Empirical Methods in Natural Language Processing, pages 2200–2009, Copenhagen, Denmark. ArXiv: 1701.03185.
- Shen et al. (2017) Xiaoyu Shen, Hui Su, Yanran Li, Wenjie Li, Shuzi Niu, Yang Zhao, Akiko Aizawa, and Guoping Long. 2017. A Conditional Variational Framework for Dialog Generation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL 2017), Vancouver, Canada. ArXiv: 1705.00316.
- Shen et al. (2018) Xiaoyu Shen, Hui Su, Shuzi Niu, and Vera Demberg. 2018. Improving Variational Encoder-Decoders in Dialogue Generation. In AAAI 2018. ArXiv: 1802.02032.
- Sordoni et al. (2015) Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Margaret Mitchell, Jian-Yun Nie, Jianfeng Gao, and Bill Dolan. 2015. A neural network approach to context-sensitive generation of conversational responses. In NAACL.
- Sountsov and Sarawagi (2016) Pavel Sountsov and Sunita Sarawagi. 2016. Length bias in Encoder Decoder Models and a Case for Global Conditioning. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1516–1525, Austin, TX, USA. Association for Computational Linguistics.
- Tu et al. (2017) Zhaopeng Tu, Yang Liu, Zhengdong Lu, Xiaohua Liu, and Hang Li. 2017. Context Gates for Neural Machine Translation. Transactions of the Association for Computational Linguistics, 5:87–99.
Vinyals and Le (2015)
Oriol Vinyals and Quoc V. Le. 2015.
A neural conversational model.
ICML Deep Learning Workshop.
- Wang et al. (2017) Di Wang, Nebojsa Jojic, Chris Brockett, and Eric Nyberg. 2017. Steering output style and topic in neural response generation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2140–2150, Copenhagen, Denmark.
- Wei et al. (2017) Bolin Wei, Shuai Lu, Lili Mou, Hao Zhou, Pascal Poupart, Ge Li, and Zhi Jin. 2017. Why Do Neural Dialog Systems Generate Short and Meaningless Replies? A Comparison between Dialog and Translation. arXiv:1712.02250 [cs]. ArXiv: 1712.02250.
- Wu et al. (2018) Yu Wu, Wei Wu, Dejian Yang, Can Xu, Zhoujun Li, and Ming Zhou. 2018. Neural Response Generation with Dynamic Vocabularies. In AAAI 2018, New Orleans, LA, USA. ArXiv: 1711.11191.
- Xing et al. (2017) Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, and Wei-Ying Ma. 2017. Topic aware neural response generation. In AAAI, pages 3351–3357.
- Xu et al. (2017) Zhen Xu, Bingquan Liu, Baoxun Wang, SUN Chengjie, Xiaolong Wang, Zhuoran Wang, and Chao Qi. 2017. Neural response generation via gan with an approximate embedding layer. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 628–637.
- Yao et al. (2017) Lili Yao, Yaoyuan Zhang, Yansong Feng, Dongyan Zhao, and Rui Yan. 2017. Towards implicit content-introducing for generative short-text conversation systems. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2180–2189.
- Zhao et al. (2017) Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. 2017. Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders. In ACL, Vancouver, Canada. ArXiv: 1703.10960.
Appendix A Determining Cut-off Coherence Score
As shown in Figure 5, the scores on the training set roughly follow a normal distribution with a slight tail on the negatively correlated side. We make the assumption that the data fit a normal distribution and estimate parameters and . We set the cut-off to so that it accounts for 95% of the scores and does not severely filter the number of resulting examples in the dataset.
Appendix B Dialogue Examples from the Training Set
Tables 4-7 show dialogue examples from the training set with different coherence scores. The instances in Table 4 are strongly lexically coherent by repeating the same words in the response when coherence scores are over 0.9. When scores are between 0.7 and 0.9 (Table 5), the responses use synonyms to words in the context but still remain semantically coherent. When scores are between 0.4 and 0.6 (Table 6), the responses are coherent but very dull; they are getting incoherent when scores are lower (Table 7).
|yeah? ## you’re new in town, right?||you call this a town?|
|so you can have marie all to yourself? ## you’re so selfish, catherine.||you’ve always been selfish.|
|wait. ## where are you going? ## to find the president.||the president, he lives!|
|that’s not good! ## it ’s fine. it itches a bit at first, but then it stops.||of course it itches!|
|you stay out of this. ## so you want water, huh? ## that’s right.||well, i got water.|
|not quite yet. ## call your grandfather to pick me up. ## i want to go home.||grandpa ’s not here .|
|some kind of whisky nobody’s ever heard of. ## why don’t you bring your own bottle?||give him the best bourbon you got, hot stuff, and don’t be gone too long.|
|put your head on my shoulder? ## denny ## i just want to remember.||i don’t think my neck even bends, anymore.|
|i don’t even know where to start. ## kitchen. ## definitely the kitchen.||specifically the stove.|
|the problem is, the liquid just stays in your gut. ## i don’t know what to do.||well, obviously it’s not getting absorbed into the bloodstream.|
|i’m gonna hold it. ## take a look at it. ## make sure it was an accident.||doesn’t sound right.|
|in fact, thank you for underplaying it. ## so the boy becomes a man||it’s amazing.|
|oh, sting? ## little bit. ## can we stop at the drug store?||oh, uh, don’t worry.|
|there’s no one in there! ## it’s not gonna happen. ## it’s a private party.||nice going .|
give me that pad. ## what are you gonna do? ## just watch the old mastermind.
|what are you doing?|
|otherwise it’s all over do whatever he wants ## alright. ## so?||how much do you want?|
|i’ll let it go. ## don ’t worry. ## i’ve totally let it go.||it’s all water under the bridge.|
|i’m so sorry. ## mag? ## thank you.||i’m going to go for a walk.|
|i dont want it. ## why ## ok||you want the car.|
|he’s thinking about leaving the theatre. ## we’ve saved you a seat here.||come on. look at her hair.|