Example-Driven Intent Prediction with Observers

by   Shikib Mehri, et al.

A key challenge of dialog systems research is to effectively and efficiently adapt to new domains. A scalable paradigm for adaptation necessitates the development of generalizable models that perform well in few-shot settings. In this paper, we focus on the intent classification problem which aims to identify user intents given utterances addressed to the dialog system. We propose two approaches for improving the generalizability of utterance classification models: (1) example-driven training and (2) observers. Example-driven training learns to classify utterances by comparing to examples, thereby using the underlying encoder as a sentence similarity model. Prior work has shown that BERT-like models tend to attribute a significant amount of attention to the [CLS] token, which we hypothesize results in diluted representations. Observers are tokens that are not attended to, and are an alternative to the [CLS] token. The proposed methods attain state-of-the-art results on three intent prediction datasets (Banking, Clinc, and HWU) in both the full data and few-shot (10 examples per intent) settings. Furthermore, we demonstrate that the proposed approach can transfer to new intents and across datasets without any additional training.


page 1

page 2

page 3

page 4


A Single Example Can Improve Zero-Shot Data Generation

Sub-tasks of intent classification, such as robustness to distribution s...

CG-BERT: Conditional Text Generation with BERT for Generalized Few-shot Intent Detection

In this paper, we formulate a more realistic and difficult problem setup...

Fuzzy Classification of Multi-intent Utterances

Current intent classification approaches assign binary intent class memb...

Revisiting Mahalanobis Distance for Transformer-Based Out-of-Domain Detection

Real-life applications, heavily relying on machine learning, such as dia...

A Base Camp for Scaling AI

Modern statistical machine learning (SML) methods share a major limitati...

Efficient Intent Detection with Dual Sentence Encoders

Building conversational systems in new domains and with added functional...

User Intent Classification using Memory Networks: A Comparative Analysis for a Limited Data Scenario

In this report, we provide a comparative analysis of different technique...