Joint Multiple Intent Detection and Slot Filling via Self-distillation

08/18/2021
by   Lisong Chen, et al.
0

Intent detection and slot filling are two main tasks in natural language understanding (NLU) for identifying users' needs from their utterances. These two tasks are highly related and often trained jointly. However, most previous works assume that each utterance only corresponds to one intent, ignoring the fact that a user utterance in many cases could include multiple intents. In this paper, we propose a novel Self-Distillation Joint NLU model (SDJN) for multi-intent NLU. First, we formulate multiple intent detection as a weakly supervised problem and approach with multiple instance learning (MIL). Then, we design an auxiliary loop via self-distillation with three orderly arranged decoders: Initial Slot Decoder, MIL Intent Decoder, and Final Slot Decoder. The output of each decoder will serve as auxiliary information for the next decoder. With the auxiliary knowledge provided by the MIL Intent Decoder, we set Final Slot Decoder as the teacher model that imparts knowledge back to Initial Slot Decoder to complete the loop. The auxiliary loop enables intents and slots to guide mutually in-depth and further boost the overall NLU performance. Experimental results on two public multi-intent datasets indicate that our model achieves strong performance compared to others.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 5

page 6

research
08/26/2021

SLIM: Explicit Slot-Intent Mapping with BERT for Joint Multi-Intent Detection and Slot Filling

Utterance-level intent detection and token-level slot filling are two ke...
research
02/22/2021

Joint Intent Detection And Slot Filling Based on Continual Learning Model

Slot filling and intent detection have become a significant theme in the...
research
08/28/2023

Joint Multiple Intent Detection and Slot Filling with Supervised Contrastive Learning and Self-Distillation

Multiple intent detection and slot filling are two fundamental and cruci...
research
12/22/2018

Joint Slot Filling and Intent Detection via Capsule Neural Networks

Being able to recognize words as slots and detect the intent of an utter...
research
06/03/2021

GL-GIN: Fast and Accurate Non-Autoregressive Model for Joint Multiple Intent Detection and Slot Filling

Multi-intent SLU can handle multiple intents in an utterance, which has ...
research
04/23/2019

Natural Language Interactions in Autonomous Vehicles: Intent Detection and Slot Filling from Passenger Utterances

Understanding passenger intents and extracting relevant slots are import...
research
05/27/2016

Stacking With Auxiliary Features

Ensembling methods are well known for improving prediction accuracy. How...

Please sign up or login with your details

Forgot password? Click here to reset