Catch Me If You Can: Identifying Fraudulent Physician Reviews with Large Language Models Using Generative Pre-Trained Transformers

04/19/2023
by   Aishwarya Deep Shukla, et al.
0

The proliferation of fake reviews of doctors has potentially detrimental consequences for patient well-being and has prompted concern among consumer protection groups and regulatory bodies. Yet despite significant advancements in the fields of machine learning and natural language processing, there remains limited comprehension of the characteristics differentiating fraudulent from authentic reviews. This study utilizes a novel pre-labeled dataset of 38048 physician reviews to establish the effectiveness of large language models in classifying reviews. Specifically, we compare the performance of traditional ML models, such as logistic regression and support vector machines, to generative pre-trained transformer models. Furthermore, we use GPT4, the newest model in the GPT family, to uncover the key dimensions along which fake and genuine physician reviews differ. Our findings reveal significantly superior performance of GPT-3 over traditional ML models in this context. Additionally, our analysis suggests that GPT3 requires a smaller training sample than traditional models, suggesting its appropriateness for tasks with scarce training data. Moreover, the superiority of GPT3 performance increases in the cold start context i.e., when there are no prior reviews of a doctor. Finally, we employ GPT4 to reveal the crucial dimensions that distinguish fake physician reviews. In sharp contrast to previous findings in the literature that were obtained using simulated data, our findings from a real-world dataset show that fake reviews are generally more clinically detailed, more reserved in sentiment, and have better structure and grammar than authentic ones.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/03/2023

Bengali Fake Reviews: A Benchmark Dataset and Detection System

The proliferation of fake reviews on various online platforms has create...
research
10/02/2019

The merits of Universal Language Model Fine-tuning for Small Datasets – a case with Dutch book reviews

We evaluated the effectiveness of using language models, that were pre-t...
research
10/08/2020

Fake Reviews Detection through Analysis of Linguistic Features

Online reviews play an integral part for success or failure of businesse...
research
05/31/2022

Uzbek Sentiment Analysis based on local Restaurant Reviews

Extracting useful information for sentiment analysis and classification ...
research
01/08/2023

Mitigating Human and Computer Opinion Fraud via Contrastive Learning

We introduce the novel approach towards fake text reviews detection in c...
research
04/15/2020

Sentiment Analysis of Yelp Reviews: A Comparison of Techniques and Models

We use over 350,000 Yelp reviews on 5,000 restaurants to perform an abla...
research
06/25/2023

Revolutionizing Cyber Threat Detection with Large Language Models

Natural Language Processing (NLP) domain is experiencing a revolution du...

Please sign up or login with your details

Forgot password? Click here to reset