DeepAI AI Chat
Log In Sign Up

Response Generation with Context-Aware Prompt Learning

by   Xiaodong Gu, et al.

Pre-trained language models (PLM) have marked a huge leap in neural dialogue modeling. While PLMs are pre-trained on large-scale text corpora, they are usually fine-tuned on scarce dialogue data with specific domain knowledge and dialogue styles. However, tailoring the language models while fully utilizing prior knowledge in large pre-trained models remains a challenge. In this paper, we present a novel approach for pre-trained dialogue modeling that casts the dialogue generation problem as a prompt-learning task. Instead of fine-tuning on limited dialogue data, our approach, DialogPrompt, learns continuous prompt embeddings optimized for dialogue contexts, which appropriately elicit knowledge from the large pre-trained model. To encourage the model to better utilize the prompt embeddings, the prompt encoders are designed to be dynamically generated based on the dialogue context. Experiments on popular conversation datasets show that our approach significantly outperforms the fine-tuning baseline and the generic prompt-learning methods. Furthermore, human evaluations strongly support the superiority of DialogPrompt in regard to response generation quality.


page 1

page 2

page 3

page 4


Are Pre-trained Language Models Knowledgeable to Ground Open Domain Dialogues?

We study knowledge-grounded dialogue generation with pre-trained languag...

Task-Oriented Dialogue System as Natural Language Generation

In this paper, we propose to formulate the task-oriented dialogue system...

Language Model as an Annotator: Exploring DialoGPT for Dialogue Summarization

Current dialogue summarization systems usually encode the text with a nu...

Injecting Domain Knowledge in Language Models for Task-Oriented Dialogue Systems

Pre-trained language models (PLM) have advanced the state-of-the-art acr...

Learning Locality and Isotropy in Dialogue Modeling

Existing dialogue modeling methods have achieved promising performance o...

Generating Repetitions with Appropriate Repeated Words

A repetition is a response that repeats words in the previous speaker's ...

Building a Personalized Dialogue System with Prompt-Tuning

Dialogue systems without consistent responses are not fascinating. In th...