Småprat: DialoGPT for Natural Language Generation of Swedish Dialogue by Transfer Learning

10/12/2021
by   Tosin Adewumi, et al.
0

Building open-domain conversational systems (or chatbots) that produce convincing responses is a recognized challenge. Recent state-of-the-art (SoTA) transformer-based models for the generation of natural language dialogue have demonstrated impressive performance in simulating human-like, single-turn conversations in English. This work investigates, by an empirical study, the potential for transfer learning of such models to Swedish language. DialoGPT, an English language pre-trained model, is adapted by training on three different Swedish language conversational datasets obtained from publicly available sources. Perplexity score (an automated intrinsic language model metric) and surveys by human evaluation were used to assess the performances of the fine-tuned models, with results that indicate that the capacity for transfer learning can be exploited with considerable success. Human evaluators asked to score the simulated dialogue judged over 57 to be human-like for the model trained on the largest (Swedish) dataset. We provide the demos and model checkpoints of our English and Swedish chatbots on the HuggingFace platform for public use.

READ FULL TEXT
research
05/07/2022

Vector Representations of Idioms in Conversational Systems

We demonstrate, in this study, that an open-domain conversational system...
research
01/23/2019

TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents

We introduce a new approach to generative data-driven dialogue systems (...
research
04/08/2021

Grapheme-to-Phoneme Transformer Model for Transfer Learning Dialects

Grapheme-to-Phoneme (G2P) models convert words to their phonetic pronunc...
research
11/15/2021

Say What? Collaborative Pop Lyric Generation Using Multitask Transfer Learning

Lyric generation is a popular sub-field of natural language generation t...
research
09/29/2020

The design and implementation of Language Learning Chatbot with XAI using Ontology and Transfer Learning

In this paper, we proposed a transfer learning-based English language le...
research
08/16/2023

MDDial: A Multi-turn Differential Diagnosis Dialogue Dataset with Reliability Evaluation

Dialogue systems for Automatic Differential Diagnosis (ADD) have a wide ...
research
04/12/2021

Building a Swedish Open-Domain Conversational Language Model

We present on-going work of evaluating the, to our knowledge, first larg...

Please sign up or login with your details

Forgot password? Click here to reset