DeepAI
Log In Sign Up

Multi-Referenced Training for Dialogue Response Generation

09/15/2020
by   Tianyu Zhao, et al.
0

In open-domain dialogue response generation, a dialogue context can be continued with diverse responses, and the dialogue models should capture such one-to-many relations. In this work, we first analyze the training objective of dialogue models from the view of Kullback-Leibler divergence (KLD) and show that the gap between the real world probability distribution and the single-referenced data's probability distribution prevents the model from learning the one-to-many relations efficiently. Then we explore approaches to multi-referenced training in two aspects. Data-wise, we generate diverse pseudo references from a powerful pretrained model to build multi-referenced data that provides a better approximation of the real-world distribution. Model-wise, we propose to equip variational models with an expressive prior, named linear Gaussian model (LGM). Experimental results of automated evaluation and human evaluation show that the methods yield significant improvements over baselines. We will release our code and data in https://github.com/ZHAOTING/dialog-processing.

READ FULL TEXT

page 1

page 2

page 3

page 4

04/10/2020

Designing Precise and Robust Dialogue Response Evaluators

Automatic dialogue response evaluator has been proposed as an alternativ...
05/31/2021

Learning from Perturbations: Diverse and Informative Dialogue Generation with Inverse Adversarial Training

In this paper, we propose Inverse Adversarial Training (IAT) algorithm f...
03/16/2022

Multi-Stage Prompting for Knowledgeable Dialogue Generation

Existing knowledge-grounded dialogue systems typically use finetuned ver...
01/05/2019

Addressing Objects and Their Relations: The Conversational Entity Dialogue Model

Statistical spoken dialogue systems usually rely on a single- or multi-d...
07/26/2019

Multi-turn Dialogue Response Generation with Autoregressive Transformer Models

Neural dialogue models, despite their successes, still suffer from lack ...
08/27/2020

Improvement of a dedicated model for open domain persona-aware dialogue generation

This paper analyzes some speed and performance improvement methods of Tr...
09/16/2020

Group-wise Contrastive Learning for Neural Dialogue Generation

Neural dialogue response generation has gained much popularity in recent...