Modeling Token-level Uncertainty to Learn Unknown Concepts in SLU via Calibrated Dirichlet Prior RNN

10/16/2020
by   Yilin Shen, et al.
0

One major task of spoken language understanding (SLU) in modern personal assistants is to extract semantic concepts from an utterance, called slot filling. Although existing slot filling models attempted to improve extracting new concepts that are not seen in training data, the performance in practice is still not satisfied. Recent research collected question and answer annotated data to learn what is unknown and should be asked, yet not practically scalable due to the heavy data collection effort. In this paper, we incorporate softmax-based slot filling neural architectures to model the sequence uncertainty without question supervision. We design a Dirichlet Prior RNN to model high-order uncertainty by degenerating as softmax layer for RNN model training. To further enhance the uncertainty modeling robustness, we propose a novel multi-task training to calibrate the Dirichlet concentration parameters. We collect unseen concepts to create two test datasets from SLU benchmark datasets Snips and ATIS. On these two and another existing Concept Learning benchmark datasets, we show that our approach significantly outperforms state-of-the-art approaches by up to 8.18 applied to any RNN or Transformer based slot filling models with a softmax layer.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/12/2018

Recurrent Neural Networks with Pre-trained Language Model Embedding for Slot Filling Task

In recent years, Recurrent Neural Networks (RNNs) based models have been...
research
11/04/2018

Elastic CRFs for Open-ontology Slot Filling

Slot filling is a crucial component in task-oriented dialog systems, whi...
research
05/18/2023

Generalized Multiple Intent Conditioned Slot Filling

Natural language understanding includes the tasks of intent detection (i...
research
11/30/2018

Inferring Concept Prerequisite Relations from Online Educational Resources

The Internet has rich and rapidly increasing sources of high quality edu...
research
09/18/2018

User Information Augmented Semantic Frame Parsing using Coarse-to-Fine Neural Networks

Semantic frame parsing is a crucial component in spoken language underst...
research
08/24/2022

PSSAT: A Perturbed Semantic Structure Awareness Transferring Method for Perturbation-Robust Slot Filling

Most existing slot filling models tend to memorize inherent patterns of ...
research
10/15/2019

Iterative Delexicalization for Improved Spoken Language Understanding

Recurrent neural network (RNN) based joint intent classification and slo...

Please sign up or login with your details

Forgot password? Click here to reset