Pre-Trained Neural Language Models for Automatic Mobile App User Feedback Answer Generation

02/04/2022
by   Yue Cao, et al.
0

Studies show that developers' answers to the mobile app users' feedbacks on app stores can increase the apps' star rating. To help app developers generate answers that are related to the users' issues, recent studies develop models to generate the answers automatically. Aims: The app response generation models use deep neural networks and require training data. Pre-Trained neural language Models (PTM) used in Natural Language Processing (NLP) take advantage of the information they learned from a large corpora in an unsupervised manner, and can reduce the amount of required training data. In this paper, we evaluate PTMs to generate replies to the mobile app user feedbacks. Method: We train a Transformer model from scratch and fine-tune two PTMs to evaluate the generated responses, which are compared to RRGEN, a current app response model. We also evaluate the models with different portions of the training data. Results: The results on a large dataset evaluated by automatic metrics show that PTMs obtain lower scores than the baselines. However, our human evaluation confirms that PTMs can generate more relevant and meaningful responses to the posted feedbacks. Moreover, the performance of PTMs has less drop compared to other models when the amount of training data is reduced to 1/3. Conclusion: PTMs are useful in generating responses to app reviews and are more robust models to the amount of training data provided. However, the prediction time is 19X than RRGEN. This study can provide new avenues for research in adapting the PTMs for analyzing mobile app user feedbacks. Index Terms-mobile app user feedback analysis, neural pre-trained language models, automatic answer generation

READ FULL TEXT

page 1

page 4

research
04/12/2021

Evaluating Pre-Trained Models for User Feedback Analysis in Software Engineering: A Study on Classification of App-Reviews

Context: Mobile app reviews written by users on app stores or social med...
research
08/28/2019

On building an automated responding system for app reviews: What are the characteristics of reviews and their responses?

Recent studies showed that the dialogs between app developers and app us...
research
10/14/2020

Learning Improvised Chatbots from Adversarial Modifications of Natural Language Feedback

The ubiquitous nature of chatbots and their interaction with users gener...
research
05/05/2022

Assistive Recipe Editing through Critiquing

There has recently been growing interest in the automatic generation of ...
research
10/04/2021

The state-of-the-art in text-based automatic personality prediction

Personality detection is an old topic in psychology and Automatic Person...
research
07/10/2023

ChatGPT for Digital Forensic Investigation: The Good, The Bad, and The Unknown

The disruptive application of ChatGPT (GPT-3.5, GPT-4) to a variety of d...
research
10/19/2018

A neural network to classify metaphorical violence on cable news

I present here an experimental system for identifying and annotating met...

Please sign up or login with your details

Forgot password? Click here to reset