Reproducibility, Replicability and Beyond: Assessing Production Readiness of Aspect Based Sentiment Analysis in the Wild

by   Rajdeep Mukherjee, et al.

With the exponential growth of online marketplaces and user-generated content therein, aspect-based sentiment analysis has become more important than ever. In this work, we critically review a representative sample of the models published during the past six years through the lens of a practitioner, with an eye towards deployment in production. First, our rigorous empirical evaluation reveals poor reproducibility: an average 4-5 sample. Second, to further bolster our confidence in empirical evaluation, we report experiments on two challenging data slices, and observe a consistent 12-55 domains and observe that as little as 10-25 dataset, when used in conjunction with datasets from other domains within the same locale, largely closes the gap between complete cross-domain and complete in-domain predictive performance. Lastly, we open-source two large-scale annotated review corpora from a large e-commerce portal in India in order to aid the study of replicability and transfer, with the hope that it will fuel further growth of the field.


page 1

page 2

page 3

page 4


ASAP: A Chinese Review Dataset Towards Aspect Category Sentiment Analysis and Rating Prediction

Sentiment analysis has attracted increasing attention in e-commerce. The...

Zero-Shot Aspect-Based Sentiment Analysis

Aspect-based sentiment analysis (ABSA) typically requires in-domain anno...

Weighed Domain-Invariant Representation Learning for Cross-domain Sentiment Analysis

Cross-domain sentiment analysis is currently a hot topic in the research...

A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challenges

As an important fine-grained sentiment analysis problem, aspect-based se...

PELESent: Cross-domain polarity classification using distant supervision

The enormous amount of texts published daily by Internet users has foste...

Cross-domain Aspect Category Transfer and Detection via Traceable Heterogeneous Graph Representation Learning

Aspect category detection is an essential task for sentiment analysis an...