Assessment of Massively Multilingual Sentiment Classifiers

04/11/2022
by   Krzysztof Rajda, et al.
1

Models are increasing in size and complexity in the hunt for SOTA. But what if those 2% increase in performance does not make a difference in a production use case? Maybe benefits from a smaller, faster model outweigh those slight performance gains. Also, equally good performance across languages in multilingual tasks is more important than SOTA results on a single one. We present the biggest, unified, multilingual collection of sentiment analysis datasets. We use these to assess 11 models and 80 high-quality sentiment datasets (out of 342 raw datasets collected) in 27 languages and included results on the internally annotated datasets. We deeply evaluate multiple setups, including fine-tuning transformer-based models for measuring performance. We compare results in numerous dimensions addressing the imbalance in both languages coverage and dataset sizes. Finally, we present some best practices for working with such a massive collection of datasets and models from a multilingual perspective.

READ FULL TEXT

page 7

page 16

research
04/27/2023

UIO at SemEval-2023 Task 12: Multilingual fine-tuning for sentiment classification in low-resource languages

Our contribution to the 2023 AfriSenti-SemEval shared task 12: Sentiment...
research
06/13/2023

Massively Multilingual Corpus of Sentiment Datasets and Multi-faceted Sentiment Classification Benchmark

Despite impressive advancements in multilingual corpora collection and m...
research
05/04/2023

DN at SemEval-2023 Task 12: Low-Resource Language Text Classification via Multilingual Pretrained Language Model Fine-tuning

In recent years, sentiment analysis has gained significant importance in...
research
08/24/2021

Are the Multilingual Models Better? Improving Czech Sentiment with Transformers

In this paper, we aim at improving Czech sentiment with transformer-base...
research
04/24/2023

KInITVeraAI at SemEval-2023 Task 3: Simple yet Powerful Multilingual Fine-Tuning for Persuasion Techniques Detection

This paper presents the best-performing solution to the SemEval 2023 Tas...
research
04/08/2023

tmn at SemEval-2023 Task 9: Multilingual Tweet Intimacy Detection using XLM-T, Google Translate, and Ensemble Learning

The paper describes a transformer-based system designed for SemEval-2023...
research
02/27/2023

Quantifying Valence and Arousal in Text with Multilingual Pre-trained Transformers

The analysis of emotions expressed in text has numerous applications. In...

Please sign up or login with your details

Forgot password? Click here to reset