HAN: Higher-order Attention Network for Spoken Language Understanding

08/26/2021
by   Dongsheng Chen, et al.
0

Spoken Language Understanding (SLU), including intent detection and slot filling, is a core component in human-computer interaction. The natural attributes of the relationship among the two subtasks make higher requirements on fine-grained feature interaction, i.e., the token-level intent features and slot features. Previous works mainly focus on jointly modeling the relationship between the two subtasks with attention-based models, while ignoring the exploration of attention order. In this paper, we propose to replace the conventional attention with our proposed Bilinear attention block and show that the introduced Higher-order Attention Network (HAN) brings improvement for the SLU task. Importantly, we conduct wide analysis to explore the effectiveness brought from the higher-order attention.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/18/2021

Towards Joint Intent Detection and Slot Filling via Higher-order Attention

Intent detection (ID) and Slot filling (SF) are two major tasks in spoke...
research
11/22/2022

A Scope Sensitive and Result Attentive Model for Multi-Intent Spoken Language Understanding

Multi-Intent Spoken Language Understanding (SLU), a novel and more compl...
research
04/21/2020

TD-GIN: Token-level Dynamic Graph-Interactive Network for Joint Multiple Intent Detection and Slot Filling

Intent detection and slot filling are two main tasks for building a spok...
research
03/20/2020

Parallel Intent and Slot Prediction using MLB Fusion

Intent and Slot Identification are two important tasks in Spoken Languag...
research
12/21/2022

Spoken Language Understanding for Conversational AI: Recent Advances and Future Direction

When a human communicates with a machine using natural language on the w...
research
09/16/2019

CM-Net: A Novel Collaborative Memory Network for Spoken Language Understanding

Spoken Language Understanding (SLU) mainly involves two tasks, intent de...
research
06/11/2021

CONDA: a CONtextual Dual-Annotated dataset for in-game toxicity understanding and detection

Traditional toxicity detection models have focused on the single utteran...

Please sign up or login with your details

Forgot password? Click here to reset