What Emotions Make One or Five Stars? Understanding Ratings of Online Product Reviews by Sentiment Analysis and XAI

02/29/2020
by   Chaehan So, et al.
0

When people buy products online, they primarily base their decisions on the recommendations of others given in online reviews. The current work analyzed these online reviews by sentiment analysis and used the extracted sentiments as features to predict the product ratings by several machine learning algorithms. These predictions were disentangled by various meth-ods of explainable AI (XAI) to understand whether the model showed any bias during prediction. Study 1 benchmarked these algorithms (knn, support vector machines, random forests, gradient boosting machines, XGBoost) and identified random forests and XGBoost as best algorithms for predicting the product ratings. In Study 2, the analysis of global feature importance identified the sentiment joy and the emotional valence negative as most predictive features. Two XAI visualization methods, local feature attributions and partial dependency plots, revealed several incorrect prediction mechanisms on the instance-level. Performing the benchmarking as classification, Study 3 identified a high no-information rate of 64.4 identified problems. In conclusion, good performance by machine learning algorithms must be taken with caution because the dataset, as encountered in this work, could be biased towards certain predictions. This work demonstrates how XAI methods reveal such prediction bias.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/03/2020

Understanding the Prediction Mechanism of Sentiments by XAI Visualization

People often rely on online reviews to make purchase decisions. The pres...
research
02/29/2020

Who Wins the Game of Thrones? How Sentiments Improve the Prediction of Candidate Choice

This paper analyzes how candidate choice prediction improves by differen...
research
01/09/2019

Sentiment Analysis of Czech Texts: An Algorithmic Survey

In the area of online communication, commerce and transactions, analyzin...
research
06/17/2020

Comparative Sentiment Analysis of App Reviews

Google app market captures the school of thought of users via ratings an...
research
03/21/2020

Sentiment Analysis in Drug Reviews using Supervised Machine Learning Algorithms

Sentiment Analysis is an important algorithm in Natural Language Process...
research
05/10/2021

Accountable Error Characterization

Customers of machine learning systems demand accountability from the com...

Please sign up or login with your details

Forgot password? Click here to reset