DeepAI AI Chat
Log In Sign Up

Response Generation with Context-Aware Prompt Learning

11/04/2021
by   Xiaodong Gu, et al.
NAVER Corp.
0

Pre-trained language models (PLM) have marked a huge leap in neural dialogue modeling. While PLMs are pre-trained on large-scale text corpora, they are usually fine-tuned on scarce dialogue data with specific domain knowledge and dialogue styles. However, tailoring the language models while fully utilizing prior knowledge in large pre-trained models remains a challenge. In this paper, we present a novel approach for pre-trained dialogue modeling that casts the dialogue generation problem as a prompt-learning task. Instead of fine-tuning on limited dialogue data, our approach, DialogPrompt, learns continuous prompt embeddings optimized for dialogue contexts, which appropriately elicit knowledge from the large pre-trained model. To encourage the model to better utilize the prompt embeddings, the prompt encoders are designed to be dynamically generated based on the dialogue context. Experiments on popular conversation datasets show that our approach significantly outperforms the fine-tuning baseline and the generic prompt-learning methods. Furthermore, human evaluations strongly support the superiority of DialogPrompt in regard to response generation quality.

READ FULL TEXT

page 1

page 2

page 3

page 4

11/19/2020

Are Pre-trained Language Models Knowledgeable to Ground Open Domain Dialogues?

We study knowledge-grounded dialogue generation with pre-trained languag...
08/31/2021

Task-Oriented Dialogue System as Natural Language Generation

In this paper, we propose to formulate the task-oriented dialogue system...
05/26/2021

Language Model as an Annotator: Exploring DialoGPT for Dialogue Summarization

Current dialogue summarization systems usually encode the text with a nu...
12/15/2022

Injecting Domain Knowledge in Language Models for Task-Oriented Dialogue Systems

Pre-trained language models (PLM) have advanced the state-of-the-art acr...
05/29/2022

Learning Locality and Isotropy in Dialogue Modeling

Existing dialogue modeling methods have achieved promising performance o...
07/03/2022

Generating Repetitions with Appropriate Repeated Words

A repetition is a response that repeats words in the previous speaker's ...
06/11/2022

Building a Personalized Dialogue System with Prompt-Tuning

Dialogue systems without consistent responses are not fascinating. In th...