DeepAI
Log In Sign Up

Diversifying Neural Dialogue Generation via Negative Distillation

05/05/2022
by   Yiwei Li, et al.
0

Generative dialogue models suffer badly from the generic response problem, limiting their applications to a few toy scenarios. Recently, an interesting approach, namely negative training, has been proposed to alleviate this problem by reminding the model not to generate high-frequency responses during training. However, its performance is hindered by two issues, ignoring low-frequency but generic responses and bringing low-frequency but meaningless responses. In this paper, we propose a novel negative training paradigm, called negative distillation, to keep the model away from the undesirable generic responses while avoiding the above problems. First, we introduce a negative teacher model that can produce query-wise generic responses, and then the student model is required to maximize the distance with multi-level negative knowledge. Empirical results show that our method outperforms previous negative training methods significantly.

READ FULL TEXT

page 1

page 2

page 3

page 4

03/06/2019

Negative Training for Neural Dialogue Response Generation

Although deep learning models have brought tremendous advancements to th...
02/22/2021

Multi-View Feature Representation for Dialogue Generation with Bidirectional Distillation

Neural dialogue models suffer from low-quality responses when interacted...
11/24/2018

Strategy of the Negative Sampling for Training Retrieval-Based Dialogue Systems

The article describes the new approach for quality improvement of automa...
02/26/2019

Generative Visual Dialogue System via Adaptive Reasoning and Weighted Likelihood Estimation

The key challenge of generative Visual Dialogue (VD) systems is to respo...
10/22/2022

Transformer-Based Conditioned Variational Autoencoder for Dialogue Generation

In human dialogue, a single query may elicit numerous appropriate respon...
09/19/2020

Enhancing Dialogue Generation via Multi-Level Contrastive Learning

Most of the existing works for dialogue generation are data-driven model...
09/14/2021

Challenging Instances are Worth Learning: Generating Valuable Negative Samples for Response Selection Training

Retrieval-based chatbot selects the appropriate response from candidates...