Log In Sign Up

Revisiting Popularity and Demographic Biases in Recommender Evaluation and Effectiveness

by   Nicola Neophytou, et al.

Recommendation algorithms are susceptible to popularity bias: a tendency to recommend popular items even when they fail to meet user needs. A related issue is that the recommendation quality can vary by demographic groups. Marginalized groups or groups that are under-represented in the training data may receive less relevant recommendations from these algorithms compared to others. In a recent study, Ekstrand et al. investigate how recommender performance varies according to popularity and demographics, and find statistically significant differences in recommendation utility between binary genders on two datasets, and significant effects based on age on one dataset. Here we reproduce those results and extend them with additional analyses. We find statistically significant differences in recommender performance by both age and gender. We observe that recommendation utility steadily degrades for older users, and is lower for women than men. We also find that the utility is higher for users from countries with more representation in the dataset. In addition, we find that total usage and the popularity of consumed content are strong predictors of recommender performance and also vary significantly across demographic groups.


page 1

page 2

page 3

page 4


The Impact of Popularity Bias on Fairness and Calibration in Recommendation

Recently there has been a growing interest in fairness-aware recommender...

The Unfairness of Popularity Bias in Music Recommendation: A Reproducibility Study

Research has shown that recommender systems are typically biased towards...

Unfair Exposure of Artists in Music Recommendation

Fairness in machine learning has been studied by many researchers. In pa...

The Unfairness of Popularity Bias in Book Recommendation

Recent studies have shown that recommendation systems commonly suffer fr...

Analyzing Item Popularity Bias of Music Recommender Systems: Are Different Genders Equally Affected?

Several studies have identified discrepancies between the popularity of ...

A Re-visit of the Popularity Baseline in Recommender Systems

Popularity is often included in experimental evaluation to provide a ref...

BoostJet: Towards Combining Statistical Aggregates with Neural Embeddings for Recommendations

Recommenders have become widely popular in recent years because of their...