Teaching Models new APIs: Domain-Agnostic Simulators for Task Oriented Dialogue

10/13/2021
by   Moya Chen, et al.
0

We demonstrate that large language models are able to simulate Task Oriented Dialogues in novel domains, provided only with an API implementation and a list of goals. We show these simulations can formulate online, automatic metrics that correlate well with human evaluations. Furthermore, by checking for whether the User's goals are met, we can use simulation to repeatedly generate training data and improve the quality of simulations themselves. With no human intervention or domain-specific training data, our simulations bootstrap end-to-end models which achieve a 37% error reduction in previously unseen domains. By including as few as 32 domain-specific conversations, bootstrapped models can match the performance of a fully-supervised model with 10× more data. To our knowledge, this is the first time simulations have been shown to be effective at bootstrapping models without explicitly requiring any domain-specific training data, rule-engineering, or humans-in-the-loop.

READ FULL TEXT
research
04/23/2020

Dynamic Fusion Network for Multi-Domain End-to-end Task-Oriented Dialog

Recent studies have shown remarkable success in end-to-end task-oriented...
research
02/14/2023

SwitchPrompt: Learning Domain-Specific Gated Soft Prompts for Classification in Low-Resource Domains

Prompting pre-trained language models leads to promising results across ...
research
08/03/2023

Domain specificity and data efficiency in typo tolerant spell checkers: the case of search in online marketplaces

Typographical errors are a major source of frustration for visitors of o...
research
02/01/2021

Automatic Expansion of Domain-Specific Affective Models for Web Intelligence Applications

Sentic computing relies on well-defined affective models of different co...
research
06/16/2021

Domain-independent User Simulation with Transformers for Task-oriented Dialogue Systems

Dialogue policy optimisation via reinforcement learning requires a large...
research
10/15/2021

DS-TOD: Efficient Domain Specialization for Task Oriented Dialog

Recent work has shown that self-supervised dialog-specific pretraining o...

Please sign up or login with your details

Forgot password? Click here to reset