DeepAI AI Chat
Log In Sign Up

Hierarchical Modeling for Out-of-Scope Domain and Intent Classification

by   Pengfei Liu, et al.

User queries for a real-world dialog system may sometimes fall outside the scope of the system's capabilities, but appropriate system responses will enable smooth processing throughout the human-computer interaction. This paper is concerned with the user's intent, and focuses on out-of-scope intent classification in dialog systems. Although user intents are highly correlated with the application domain, few studies have exploited such correlations for intent classification. Rather than developing a two-stage approach that first classifies the domain and then the intent, we propose a hierarchical multi-task learning approach based on a joint model to classify domain and intent simultaneously. Novelties in the proposed approach include: (1) sharing supervised out-of-scope signals in joint modeling of domain and intent classification to replace a two-stage pipeline; and (2) introducing a hierarchical model that learns the intent and domain representations in the higher and lower layers respectively. Experiments show that the model outperforms existing methods in terms of accuracy, out-of-scope recall and F1. Additionally, threshold-based post-processing further improves performance by balancing precision and recall in intent classification.


page 1

page 2

page 3

page 4


An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction

Task-oriented dialog systems need to know when a query falls outside the...

Seq2Seq and Multi-Task Learning for joint intent and content extraction for domain specific interpreters

This study evaluates the performances of an LSTM network for detecting a...

Exploring the Advantages of Dense-Vector to One-Hot Encoding of Intent Classes in Out-of-Scope Detection Tasks

This work explores the intrinsic limitations of the popular one-hot enco...

Bootstrapping NLU Models with Multi-task Learning

Bootstrapping natural language understanding (NLU) systems with minimal ...

An Open-World Lottery Ticket for Out-of-Domain Intent Classification

Most existing methods of Out-of-Domain (OOD) intent classification, whic...