Log In Sign Up

ConceptNet infused DialoGPT for Underlying Commonsense Understanding and Reasoning in Dialogue Response Generation

by   Ye Liu, et al.

The pre-trained conversational models still fail to capture the implicit commonsense (CS) knowledge hidden in the dialogue interaction, even though they were pre-trained with an enormous dataset. In order to build a dialogue agent with CS capability, we firstly inject external knowledge into a pre-trained conversational model to establish basic commonsense through efficient Adapter tuning (Section 4). Secondly, we propose the “two-way learning” method to enable the bidirectional relationship between CS knowledge and sentence pairs so that the model can generate a sentence given the CS triplets, also generate the underlying CS knowledge given a sentence (Section 5). Finally, we leverage this integrated CS capability to improve open-domain dialogue response generation so that the dialogue agent is capable of understanding the CS knowledge hidden in dialogue history on top of inferring related other knowledge to further guide response generation (Section 6). The experiment results demonstrate that CS_Adapter fusion helps DialoGPT to be able to generate series of CS knowledge. And the DialoGPT+CS_Adapter response model adapted from CommonGen training can generate underlying CS triplets that fits better to dialogue context.


page 1

page 2

page 3

page 4


Empathetic Dialogue Generation with Pre-trained RoBERTa-GPT2 and External Knowledge

One challenge for dialogue agents is to recognize feelings of the conver...

DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation

We present a large, tunable neural conversational response generation mo...

Commonsense Knowledge in Wikidata

Wikidata and Wikipedia have been proven useful for reason-ing in natural...

Language Model as an Annotator: Exploring DialoGPT for Dialogue Summarization

Current dialogue summarization systems usually encode the text with a nu...

Conversational Semantic Role Labeling

Semantic role labeling (SRL) aims to extract the arguments for each pred...

Mind the Gap! Injecting Commonsense Knowledge for Abstractive Dialogue Summarization

In this paper, we propose to leverage the unique characteristics of dial...

Unsupervised Injection of Knowledge into Dialogue Generation via Language Models

Neural conversation models have shown the power to produce more meaningf...