SOLOIST: Few-shot Task-Oriented Dialog with A Single Pre-trained Auto-regressive Model

05/11/2020
by   Baolin Peng, et al.
5

This paper presents a new method SOLOIST, which uses transfer learning to efficiently build task-oriented dialog systems at scale. We parameterize a dialog system using a Transformer-based auto-regressive language model, which subsumes different dialog mod-ules (e.g.,state tracker, dialog policy, responsegenerator) into a single neural model. We pre-train, on large heterogeneous dialog corpora, a large-scale Transformer model which can generate dialog responses grounded in user goals and real-world knowledge for task completion. The pre-trained model can be efficiently adapted to accomplish a new dialog task with a handful of task-specific dialogs via machine teaching. Our experiments demonstrate that (i) SOLOIST creates new state-of-the-art results on two well-known benchmarks, CamRest and MultiWOZ, (ii) in the few-shot learning setting, the dialog systems developed by SOLOIST significantly outperform those by existing methods, and (iii) the use of machine teaching substantially reduces the labeling cost. We will release our code and pre-trained models for reproducible research.

READ FULL TEXT
research
11/29/2021

GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-Supervised Learning and Explicit Policy Injection

Pre-trained models have proved to be powerful in enhancing task-oriented...
research
10/21/2021

SYNERGY: Building Task Bots at Scale Using Symbolic Knowledge and Machine Teaching

In this paper we explore the use of symbolic knowledge and machine teach...
research
01/18/2022

Toward Self-Learning End-to-End Dialog Systems

End-to-end task-oriented dialog systems often suffer from out-of-distrib...
research
03/05/2020

EmpTransfo: A Multi-head Transformer Architecture for Creating Empathetic Dialog Systems

Understanding emotions and responding accordingly is one of the biggest ...
research
09/07/2020

Robust Conversational AI with Grounded Text Generation

This article presents a hybrid approach based on a Grounded Text Generat...
research
07/10/2023

SimpleMTOD: A Simple Language Model for Multimodal Task-Oriented Dialogue with Symbolic Scene Representation

SimpleMTOD is a simple language model which recasts several sub-tasks in...
research
08/28/2021

Self-training Improves Pre-training for Few-shot Learning in Task-oriented Dialog Systems

As the labeling cost for different modules in task-oriented dialog (ToD)...

Please sign up or login with your details

Forgot password? Click here to reset