DeepAI AI Chat
Log In Sign Up

BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data

by   Haoyu Song, et al.

Maintaining consistent personas is essential for dialogue agents. Although tremendous advancements have been brought, the limited-scale of annotated persona-dense data are still barriers towards training robust and consistent persona-based dialogue models. In this work, we show how the challenges can be addressed by disentangling persona-based dialogue generation into two sub-tasks with a novel BERT-over-BERT (BoB) model. Specifically, the model consists of a BERT-based encoder and two BERT-based decoders, where one decoder is for response generation, and another is for consistency understanding. In particular, to learn the ability of consistency understanding from large-scale non-dialogue inference data, we train the second decoder in an unlikelihood manner. Under different limited data settings, both automatic and human evaluations demonstrate that the proposed model outperforms strong baselines in response quality and persona consistency.


page 1

page 2

page 3

page 4


Generating Persona Consistent Dialogues by Exploiting Natural Language Inference

Consistency is one of the major challenges faced by dialogue agents. A h...

Profile Consistency Identification for Open-domain Dialogue Agents

Maintaining a consistent attribute profile is crucial for dialogue agent...

A Model-Agnostic Data Manipulation Method for Persona-based Dialogue Generation

Towards building intelligent dialogue agents, there has been a growing i...

You Impress Me: Dialogue Generation via Mutual Persona Perception

Despite the continuing efforts to improve the engagingness and consisten...

Learning to Memorize Entailment and Discourse Relations for Persona-Consistent Dialogues

Maintaining engagement and consistency is particularly important in dial...

On Task-Adaptive Pretraining for Dialogue Response Selection

Recent advancements in dialogue response selection (DRS) are based on th...

COSPLAY: Concept Set Guided Personalized Dialogue Generation Across Both Party Personas

Maintaining a consistent persona is essential for building a human-like ...

Code Repositories


The source codes for ACL 2021 paper 'BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data'

view repo