Log In Sign Up

Diversifying Neural Dialogue Generation via Negative Distillation

by   Yiwei Li, et al.

Generative dialogue models suffer badly from the generic response problem, limiting their applications to a few toy scenarios. Recently, an interesting approach, namely negative training, has been proposed to alleviate this problem by reminding the model not to generate high-frequency responses during training. However, its performance is hindered by two issues, ignoring low-frequency but generic responses and bringing low-frequency but meaningless responses. In this paper, we propose a novel negative training paradigm, called negative distillation, to keep the model away from the undesirable generic responses while avoiding the above problems. First, we introduce a negative teacher model that can produce query-wise generic responses, and then the student model is required to maximize the distance with multi-level negative knowledge. Empirical results show that our method outperforms previous negative training methods significantly.


page 1

page 2

page 3

page 4


Negative Training for Neural Dialogue Response Generation

Although deep learning models have brought tremendous advancements to th...

Multi-View Feature Representation for Dialogue Generation with Bidirectional Distillation

Neural dialogue models suffer from low-quality responses when interacted...

Strategy of the Negative Sampling for Training Retrieval-Based Dialogue Systems

The article describes the new approach for quality improvement of automa...

Generative Visual Dialogue System via Adaptive Reasoning and Weighted Likelihood Estimation

The key challenge of generative Visual Dialogue (VD) systems is to respo...

Transformer-Based Conditioned Variational Autoencoder for Dialogue Generation

In human dialogue, a single query may elicit numerous appropriate respon...

Enhancing Dialogue Generation via Multi-Level Contrastive Learning

Most of the existing works for dialogue generation are data-driven model...

Challenging Instances are Worth Learning: Generating Valuable Negative Samples for Response Selection Training

Retrieval-based chatbot selects the appropriate response from candidates...