DeepAI AI Chat
Log In Sign Up

ASSIST: Towards Label Noise-Robust Dialogue State Tracking

by   Fanghua Ye, et al.

The MultiWOZ 2.0 dataset has greatly boosted the research on dialogue state tracking (DST). However, substantial noise has been discovered in its state annotations. Such noise brings about huge challenges for training DST models robustly. Although several refined versions, including MultiWOZ 2.1-2.4, have been published recently, there are still lots of noisy labels, especially in the training set. Besides, it is costly to rectify all the problematic annotations. In this paper, instead of improving the annotation quality further, we propose a general framework, named ASSIST (lAbel noiSe-robuSt dIalogue State Tracking), to train DST models robustly from noisy labels. ASSIST first generates pseudo labels for each sample in the training set by using an auxiliary model trained on a small clean dataset, then puts the generated pseudo labels and vanilla noisy labels together to train the primary model. We show the validity of ASSIST theoretically. Experimental results also demonstrate that ASSIST improves the joint goal accuracy of DST by up to 28.16% on the initial version MultiWOZ 2.0 and 8.41% on the latest version MultiWOZ 2.4, respectively.


page 1

page 2

page 3

page 4


MetaASSIST: Robust Dialogue State Tracking with Meta Learning

Existing dialogue datasets contain lots of noise in their state annotati...

Pseudo-Label Ensemble-based Semi-supervised Learning for Handling Noisy Soiling Segmentation Annotations

Manual annotation of soiling on surround view cameras is a very challeng...

MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselines

MultiWOZ is a well-known task-oriented dialogue dataset containing over ...

Improving group robustness under noisy labels using predictive uncertainty

The standard empirical risk minimization (ERM) can underperform on certa...

Co-sampling: Training Robust Networks for Extremely Noisy Supervision

Training robust deep networks is challenging under noisy labels. Current...

Learning functional sections in medical conversations: iterative pseudo-labeling and human-in-the-loop approach

Medical conversations between patients and medical professionals have im...