EVA2.0: Investigating Open-Domain Chinese Dialogue Systems with Large-Scale Pre-Training

03/17/2022
by   Yuxian Gu, et al.
0

Large-scale pre-training has shown remarkable performance in building open-domain dialogue systems. However, previous works mainly focus on showing and evaluating the conversational performance of the released dialogue model, ignoring the discussion of some key factors towards a powerful human-like chatbot, especially in Chinese scenarios. In this paper, we conduct extensive experiments to investigate these under-explored factors, including data quality control, model architecture designs, training approaches, and decoding strategies. We propose EVA2.0, a large-scale pre-trained open-domain Chinese dialogue model with 2.8 billion parameters, and make our models and code publicly available. To our knowledge, EVA2.0 is the largest open-source Chinese dialogue model. Automatic and human evaluations show that our model significantly outperforms other open-source counterparts. We also discuss the limitations of this work by presenting some failure cases and pose some future directions.

READ FULL TEXT
research
08/03/2021

EVA: An Open-Domain Chinese Dialogue System with Large-Scale Generative Pre-Training

Although pre-trained language models have remarkably enhanced the genera...
research
04/16/2023

ChatPLUG: Open-Domain Generative Dialogue System with Internet-Augmented Instruction Tuning for Digital Human

In this paper, we present ChatPLUG, a Chinese open-domain dialogue syste...
research
04/28/2020

Recipes for building an open-domain chatbot

Building open-domain chatbots is a challenging area for machine learning...
research
04/16/2023

Towards Better Instruction Following Language Models for Chinese: Investigating the Impact of Training Data and Evaluation

Recently, significant public efforts have been directed towards developi...
research
08/30/2022

Towards Boosting the Open-Domain Chatbot with Human Feedback

Many open-domain dialogue models pre-trained with social media comments ...
research
10/14/2020

Recipes for Safety in Open-domain Chatbots

Models trained on large unlabeled corpora of human interactions will lea...
research
09/03/2022

CrossDial: An Entertaining Dialogue Dataset of Chinese Crosstalk

Crosstalk is a traditional Chinese theatrical performance art. It is com...

Please sign up or login with your details

Forgot password? Click here to reset