Multi-Domain Dialogue State Tracking – A Purely Transformer-Based Generative Approach

10/24/2020
by   Yan Zeng, et al.
0

We investigate the problem of multi-domain Dialogue State Tracking (DST) with open vocabulary. Existing approaches exploit BERT encoder and copy-based RNN decoder, where the encoder first predicts the state operation, and then the decoder generates new slot values. However, in this stacked encoder-decoder structure, the operation prediction objective only affects the BERT encoder and the value generation objective mainly affects the RNN decoder. In this paper, we propose a purely Transformer-based framework that uses BERT as both encoder and decoder. In so doing, the operation prediction objective and the value generation objective can jointly optimize our model for DST. At the decoding step, we re-use the hidden states of the encoder in the self-attention mechanism of the corresponding decoder layer to construct a flat model structure for effective parameter updating. Experimental results show that our approach substantially outperforms the existing state-of-the-art framework, and it also achieves very competitive performance to the best ontology-based approaches.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/21/2020

Multi-Domain Dialogue State Tracking based on State Graph

We investigate the problem of multi-domain Dialogue State Tracking (DST)...
research
07/25/2021

Learn to Focus: Hierarchical Dynamic Copy Network for Dialogue State Tracking

Recently, researchers have explored using the encoder-decoder framework ...
research
11/10/2019

Efficient Dialogue State Tracking by Selectively Overwriting Memory

Recent works in dialogue state tracking (DST) focus on an open vocabular...
research
05/25/2016

Review Networks for Caption Generation

We propose a novel extension of the encoder-decoder framework, called a ...
research
09/28/2021

Nana-HDR: A Non-attentive Non-autoregressive Hybrid Model for TTS

This paper presents Nana-HDR, a new non-attentive non-autoregressive mod...
research
07/05/2019

BERT-DST: Scalable End-to-End Dialogue State Tracking with Bidirectional Encoder Representations from Transformer

An important yet rarely tackled problem in dialogue state tracking (DST)...
research
08/21/2019

Copy-Enhanced Heterogeneous Information Learning for Dialogue State Tracking

Dialogue state tracking (DST) is an essential component in task-oriented...

Please sign up or login with your details

Forgot password? Click here to reset