DSTEA: Dialogue State Tracking with Entity Adaptive Pre-training

07/08/2022
by   Yukyung Lee, et al.
0

Dialogue state tracking (DST) is a core sub-module of a dialogue system, which aims to extract the appropriate belief state (domain-slot-value) from a system and user utterances. Most previous studies have attempted to improve performance by increasing the size of the pre-trained model or using additional features such as graph relations. In this study, we propose dialogue state tracking with entity adaptive pre-training (DSTEA), a system in which key entities in a sentence are more intensively trained by the encoder of the DST model. DSTEA extracts important entities from input dialogues in four ways, and then applies selective knowledge masking to train the model effectively. Although DSTEA conducts only pre-training without directly infusing additional knowledge to the DST model, it achieved better performance than the best-known benchmark models on MultiWOZ 2.0, 2.1, and 2.2. The effectiveness of DSTEA was verified through various comparative experiments with regard to the entity type and different adaptive settings.

READ FULL TEXT
research
09/19/2022

Semantic-based Pre-training for Dialogue Understanding

Pre-trained language models have made great progress on dialogue tasks. ...
research
05/23/2021

Structural Pre-training for Dialogue Comprehension

Pre-trained language models (PrLMs) have demonstrated superior performan...
research
09/12/2021

Knowledge Enhanced Fine-Tuning for Better Handling Unseen Entities in Dialogue Generation

Although pre-training models have achieved great success in dialogue gen...
research
08/31/2021

Effective Sequence-to-Sequence Dialogue State Tracking

Sequence-to-sequence models have been applied to a wide variety of NLP t...
research
09/01/2019

Pre-training of Deep Contextualized Embeddings of Words and Entities for Named Entity Disambiguation

Deep contextualized embeddings trained using unsupervised language model...
research
10/08/2022

On Task-Adaptive Pretraining for Dialogue Response Selection

Recent advancements in dialogue response selection (DRS) are based on th...
research
10/20/2022

Tele-Knowledge Pre-training for Fault Analysis

In this work, we share our experience on tele-knowledge pre-training for...

Please sign up or login with your details

Forgot password? Click here to reset