Sentiment Analysis in the Era of Large Language Models: A Reality Check

05/24/2023
by   Wenxuan Zhang, et al.
0

Sentiment analysis (SA) has been a long-standing research area in natural language processing. It can offer rich insights into human sentiments and opinions and has thus seen considerable interest from both academia and industry. With the advent of large language models (LLMs) such as ChatGPT, there is a great potential for their employment on SA problems. However, the extent to which existing LLMs can be leveraged for different sentiment analysis tasks remains unclear. This paper aims to provide a comprehensive investigation into the capabilities of LLMs in performing various sentiment analysis tasks, from conventional sentiment classification to aspect-based sentiment analysis and multifaceted analysis of subjective texts. We evaluate performance across 13 tasks on 26 datasets and compare the results against small language models (SLMs) trained on domain-specific datasets. Our study reveals that while LLMs demonstrate satisfactory performance in simpler tasks, they lag behind in more complex tasks requiring deeper understanding or structured sentiment information. However, LLMs significantly outperform SLMs in few-shot learning settings, suggesting their potential when annotation resources are limited. We also highlight the limitations of current evaluation practices in assessing LLMs' SA abilities and propose a novel benchmark, SentiEval, for a more comprehensive and realistic evaluation. Data and code during our investigations are available at <https://github.com/DAMO-NLP-SG/LLM-Sentiment>.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/05/2023

Enhance Multi-domain Sentiment Analysis of Review Texts through Prompting Strategies

Large Language Models (LLMs) have made significant strides in both scien...
research
06/07/2023

PromptBench: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts

The increasing reliance on Large Language Models (LLMs) across academia ...
research
04/20/2023

Can ChatGPT Reproduce Human-Generated Labels? A Study of Social Computing Tasks

The release of ChatGPT has uncovered a range of possibilities whereby la...
research
05/24/2023

OverPrompt: Enhancing ChatGPT Capabilities through an Efficient In-Context Learning Approach

The exceptional performance of pre-trained large language models has rev...
research
01/06/2022

Fortunately, Discourse Markers Can Enhance Language Models for Sentiment Analysis

In recent years, pretrained language models have revolutionized the NLP ...
research
07/16/2023

SentimentGPT: Exploiting GPT for Advanced Sentiment Analysis and its Departure from Current Machine Learning

This study presents a thorough examination of various Generative Pretrai...

Please sign up or login with your details

Forgot password? Click here to reset