Recipes for building an open-domain chatbot

by   Stephen Roller, et al.

Building open-domain chatbots is a challenging area for machine learning research. While prior work has shown that scaling neural models in the number of parameters and the size of the data they are trained on gives improved results, we show that other ingredients are important for a high-performing chatbot. Good conversation requires a number of skills that an expert conversationalist blends in a seamless way: providing engaging talking points and listening to their partners, and displaying knowledge, empathy and personality appropriately, while maintaining a consistent persona. We show that large scale models can learn these skills when given appropriate training data and choice of generation strategy. We build variants of these recipes with 90M, 2.7B and 9.4B parameter models, and make our models and code publicly available under the collective name Blender. Human evaluations show our best models are superior to existing approaches in multi-turn dialogue in terms of engagingness and humanness measurements. We then discuss the limitations of this work by analyzing failure cases of our models.


page 1

page 8

page 16

page 17

page 18

page 19

page 20


EVA2.0: Investigating Open-Domain Chinese Dialogue Systems with Large-Scale Pre-Training

Large-scale pre-training has shown remarkable performance in building op...

If I Hear You Correctly: Building and Evaluating Interview Chatbots with Active Listening Skills

Interview chatbots engage users in a text-based conversation to draw out...

ChatPLUG: Open-Domain Generative Dialogue System with Internet-Augmented Instruction Tuning for Digital Human

In this paper, we present ChatPLUG, a Chinese open-domain dialogue syste...

Learning C to x86 Translation: An Experiment in Neural Compilation

Deep learning has had a significant impact on many fields. Recently, cod...

Recipes for Safety in Open-domain Chatbots

Models trained on large unlabeled corpora of human interactions will lea...

Learning New Skills after Deployment: Improving open-domain internet-driven dialogue with human feedback

Frozen models trained to mimic static datasets can never improve their p...

Neural Conversation Models and How to Rein Them in: A Survey of Failures and Fixes

Recent conditional language models are able to continue any kind of text...

Code Repositories


Machine Learning Examples I have used and in some cases created

view repo


Everything about AI ChatBot - Research, Code and Samples

view repo

Please sign up or login with your details

Forgot password? Click here to reset