A Single Example Can Improve Zero-Shot Data Generation

by   Pavel Burnyshev, et al.

Sub-tasks of intent classification, such as robustness to distribution shift, adaptation to specific user groups and personalization, out-of-domain detection, require extensive and flexible datasets for experiments and evaluation. As collecting such datasets is time- and labor-consuming, we propose to use text generation methods to gather datasets. The generator should be trained to generate utterances that belong to the given intent. We explore two approaches to generating task-oriented utterances. In the zero-shot approach, the model is trained to generate utterances from seen intents and is further used to generate utterances for intents unseen during training. In the one-shot approach, the model is presented with a single utterance from a test intent. We perform a thorough automatic, and human evaluation of the dataset generated utilizing two proposed approaches. Our results reveal that the attributes of the generated data are close to original test sets, collected via crowd-sourcing.


page 1

page 2

page 3

page 4


Zero-shot User Intent Detection via Capsule Neural Networks

User intent detection plays a critical role in question-answering and di...

Improved Goal Oriented Dialogue via Utterance Generation and Look Ahead

Goal oriented dialogue systems have become a prominent customer-care int...

Example-Driven Intent Prediction with Observers

A key challenge of dialog systems research is to effectively and efficie...

Z-BERT-A: a zero-shot Pipeline for Unknown Intent detection

Intent discovery is a fundamental task in NLP, and it is increasingly re...

Revisiting Mahalanobis Distance for Transformer-Based Out-of-Domain Detection

Real-life applications, heavily relying on machine learning, such as dia...

Template-based Approach to Zero-shot Intent Recognition

The recent advances in transfer learning techniques and pre-training of ...

Energy-based Unknown Intent Detection with Data Manipulation

Unknown intent detection aims to identify the out-of-distribution (OOD) ...

Please sign up or login with your details

Forgot password? Click here to reset