Improving Neural Response Diversity with Frequency-Aware Cross-Entropy Loss

02/25/2019
by   Shaojie Jiang, et al.
0

Sequence-to-Sequence (Seq2Seq) models have achieved encouraging performance on the dialogue response generation task. However, existing Seq2Seq-based response generation methods suffer from a low-diversity problem: they frequently generate generic responses, which make the conversation less interesting. In this paper, we address the low-diversity problem by investigating its connection with model over-confidence reflected in predicted distributions. Specifically, we first analyze the influence of the commonly used Cross-Entropy (CE) loss function, and find that the CE loss function prefers high-frequency tokens, which results in low-diversity responses. We then propose a Frequency-Aware Cross-Entropy (FACE) loss function that improves over the CE loss function by incorporating a weighting mechanism conditioned on token frequency. Extensive experiments on benchmark datasets show that the FACE loss function is able to substantially improve the diversity of existing state-of-the-art Seq2Seq response generation methods, in terms of both automatic and human evaluations.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/20/2018

Another Diversity-Promoting Objective Function for Neural Dialogue Generation

Although generation-based dialogue systems have been widely researched, ...
research
09/06/2018

Why are Sequence-to-Sequence Models So Dull? Understanding the Low-Diversity Problem of Chatbots

Diversity is a long-studied topic in information retrieval that usually ...
research
05/21/2022

CORAL: Contextual Response Retrievability Loss Function for Training Dialog Generation Models

Natural Language Generation (NLG) represents a large collection of tasks...
research
12/01/2019

HSCJN: A Holistic Semantic Constraint Joint Network for Diverse Response Generation

The sequence-to-sequence (Seq2Seq) model generates target words iterativ...
research
03/24/2022

Steganalysis of Image with Adaptively Parametric Activation

Steganalysis as a method to detect whether image contains se-cret messag...
research
01/03/2023

Effective and Efficient Training for Sequential Recommendation Using Cumulative Cross-Entropy Loss

Increasing research interests focus on sequential recommender systems, a...
research
06/20/2021

A Brief Study on the Effects of Training Generative Dialogue Models with a Semantic loss

Neural models trained for next utterance generation in dialogue task lea...

Please sign up or login with your details

Forgot password? Click here to reset