Adaptive Bridge between Training and Inference for Dialogue

10/22/2021
by   Haoran Xu, et al.
0

Although exposure bias has been widely studied in some NLP tasks, it faces its unique challenges in dialogue response generation, the representative one-to-various generation scenario. In real human dialogue, there are many appropriate responses for the same context, not only with different expressions, but also with different topics. Therefore, due to the much bigger gap between various ground-truth responses and the generated synthetic response, exposure bias is more challenging in dialogue generation task. What's more, as MLE encourages the model to only learn the common words among different ground-truth responses, but ignores the interesting and specific parts, exposure bias may further lead to the common response generation problem, such as "I don't know" and "HaHa?" In this paper, we propose a novel adaptive switching mechanism, which learns to automatically transit between ground-truth learning and generated learning regarding the word-level matching score, such as the cosine similarity. Experimental results on both Chinese STC dataset and English Reddit dataset, show that our adaptive method achieves a significant improvement in terms of metric-based evaluation and human evaluation, as compared with the state-of-the-art exposure bias approaches. Further analysis on NMT task also shows that our model can achieve a significant improvement.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 4

06/12/2020

Speaker Sensitive Response Evaluation Model

Automatic evaluation of open-domain dialogue response generation is very...
11/27/2021

Partner Personas Generation for Diverse Dialogue Generation

Incorporating personas information allows diverse and engaging responses...
10/01/2019

Generalization in Generation: A closer look at Exposure Bias

Exposure bias refers to the train-test discrepancy that seemingly arises...
09/03/2019

Adversarial Bootstrapping for Dialogue Model Training

Open domain neural dialogue models, despite their successes, are known t...
09/28/2020

Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learning

Dialogue systems play an increasingly important role in various aspects ...
05/29/2021

Maintaining Common Ground in Dynamic Environments

Common grounding is the process of creating and maintaining mutual under...
04/30/2019

Investigation of Initialization Strategies for the Multiple Instance Adaptive Cosine Estimator

Sensors which use electromagnetic induction (EMI) to excite a response i...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.