DeepAI AI Chat
Log In Sign Up

Enhancing Dialogue Generation via Multi-Level Contrastive Learning

by   Xin Li, et al.

Most of the existing works for dialogue generation are data-driven models trained directly on corpora crawled from websites. They mainly focus on improving the model architecture to produce better responses but pay little attention to considering the quality of the training data contrastively. In this paper, we propose a multi-level contrastive learning paradigm to model the fine-grained quality of the responses with respect to the query. A Rank-aware Calibration (RC) network is designed to construct the multi-level contrastive optimization objectives. Since these objectives are calculated based on the sentence level, which may erroneously encourage/suppress the generation of uninformative/informative words. To tackle this incidental issue, on one hand, we design an exquisite token-level strategy for estimating the instance loss more accurately. On the other hand, we build a Knowledge Inference (KI) component to capture the keyword knowledge from the reference during training and exploit such information to encourage the generation of informative words. We evaluate the proposed model on a carefully annotated dialogue dataset and the results suggest that our model can generate more relevant and diverse responses compared to the baseline models.


page 1

page 2

page 3

page 4


Controlling Dialogue Generation with Semantic Exemplars

Dialogue systems pretrained with large language models generate locally ...

Dynamically Retrieving Knowledge via Query Generation for informative dialogue response

Knowledge-driven dialogue generation has recently made remarkable breakt...

KddRES: A Multi-level Knowledge-driven Dialogue Dataset for Restaurant Towards Customized Dialogue System

Compared with CrossWOZ (Chinese) and MultiWOZ (English) dataset which ha...

Focus-Constrained Attention Mechanism for CVAE-based Response Generation

To model diverse responses for a given post, one promising way is to int...

Diversifying Neural Dialogue Generation via Negative Distillation

Generative dialogue models suffer badly from the generic response proble...

Grayscale Data Construction and Multi-Level Ranking Objective for Dialogue Response Selection

Response selection plays a vital role in building retrieval-based conver...

Semantic-aware Contrastive Learning for More Accurate Semantic Parsing

Since the meaning representations are detailed and accurate annotations ...