A Unified Pre-training Framework for Conversational AI

by   Siqi Bao, et al.

In this work, we explore the application of PLATO-2 on various dialogue systems, including open-domain conversation, knowledge grounded dialogue, and task-oriented conversation. PLATO-2 is initially designed as an open-domain chatbot, trained via two-stage curriculum learning. In the first stage, a coarse-grained response generation model is learned to fit the simplified one-to-one mapping relationship. This model is applied to the task-oriented conversation, given that the semantic mappings tend to be deterministic in task completion. In the second stage, another fine-grained generation model and an evaluation model are further learned for diverse response generation and coherence estimation, respectively. With superior capability on capturing one-to-many mapping, such models are suitable for the open-domain conversation and knowledge grounded dialogue. For the comprehensive evaluation of PLATO-2, we have participated in multiple tasks of DSTC9, including interactive evaluation of open-domain conversation (Track3-task2), static evaluation of knowledge grounded dialogue (Track3-task1), and end-to-end task-oriented conversation (Track2-task1). PLATO-2 has obtained the 1st place in all three tasks, verifying its effectiveness as a unified framework for various dialogue systems.


page 2

page 6


PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning

To build a high-quality open-domain chatbot, we introduce the effective ...

Prediction, Selection, and Generation: Exploration of Knowledge-Driven Conversation System

In open-domain conversational systems, it is important but challenging t...

Open Domain Dialogue Generation with Latent Images

We consider grounding open domain dialogues with images. Existing work a...

A Controllable Model of Grounded Response Generation

Current end-to-end neural conversation models inherently lack the flexib...

PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generation

To explore the limit of dialogue generation pre-training, we present the...

Alquist 4.0: Towards Social Intelligence Using Generative Models and Dialogue Personalization

The open domain-dialogue system Alquist has a goal to conduct a coherent...

Manual-Guided Dialogue for Flexible Conversational Agents

How to build and use dialogue data efficiently, and how to deploy models...