Empirical Analysis of Training Strategies of Transformer-based Japanese Chit-chat Systems

09/11/2021
by   Hiroaki Sugiyama, et al.
0

In recent years, several high-performance conversational systems have been proposed based on the Transformer encoder-decoder model. Although previous studies analyzed the effects of the model parameters and the decoding method on subjective dialogue evaluations with overall metrics, they did not analyze how the differences of fine-tuning datasets affect on user's detailed impression. In addition, the Transformer-based approach has only been verified for English, not for such languages with large inter-language distances as Japanese. In this study, we develop large-scale Transformer-based Japanese dialogue models and Japanese chit-chat datasets to examine the effectiveness of the Transformer-based approach for building chit-chat dialogue systems. We evaluated and analyzed the impressions of human dialogues in different fine-tuning datasets, model parameters, and the use of additional information.

READ FULL TEXT
research
10/16/2019

Mix-review: Alleviate Forgetting in the Pretrain-Finetune Framework for Neural Language Generation Models

In this work, we study how the large-scale pretrain-finetune framework c...
research
06/11/2022

Building a Personalized Dialogue System with Prompt-Tuning

Dialogue systems without consistent responses are not fascinating. In th...
research
10/09/2020

Plug-and-Play Conversational Models

There has been considerable progress made towards conversational models ...
research
05/06/2023

Controllable Mixed-Initiative Dialogue Generation through Prompting

Mixed-initiative dialogue tasks involve repeated exchanges of informatio...
research
02/22/2021

Transformer is All You Need: Multimodal Multitask Learning with a Unified Transformer

We propose UniT, a Unified Transformer model to simultaneously learn the...
research
10/13/2021

Federated Natural Language Generation for Personalized Dialogue System

Neural conversational models have long suffered from the problem of inco...
research
03/09/2022

PALI-NLP at SemEval-2022 Task 4: Discriminative Fine-tuning of Deep Transformers for Patronizing and Condescending Language Detection

Patronizing and condescending language (PCL) has a large harmful impact ...

Please sign up or login with your details

Forgot password? Click here to reset