Quantitative analysis of Matthew effect and sparsity problem of recommender systems

by   Hao Wang, et al.

Recommender systems have received great commercial success. Recommendation has been used widely in areas such as e-commerce, online music FM, online news portal, etc. However, several problems related to input data structure pose serious challenge to recommender system performance. Two of these problems are Matthew effect and sparsity problem. Matthew effect heavily skews recommender system output towards popular items. Data sparsity problem directly affects the coverage of recommendation result. Collaborative filtering is a simple benchmark ubiquitously adopted in the industry as the baseline for recommender system design. Understanding the underlying mechanism of collaborative filtering is crucial for further optimization. In this paper, we do a thorough quantitative analysis on Matthew effect and sparsity problem in the particular context setting of collaborative filtering. We compare the underlying mechanism of user-based and item-based collaborative filtering and give insight to industrial recommender system builders.



page 4

page 5


Movie Recommender Systems: Implementation and Performance Evaluation

Over the years, explosive growth in the number of items in the catalog o...

Deep Heterogeneous Autoencoders for Collaborative Filtering

This paper leverages heterogeneous auxiliary information to address the ...

Sparsity Normalization: Stabilizing the Expected Outputs of Deep Networks

The learning of deep models, in which a numerous of parameters are super...

Local Optimality of User Choices and Collaborative Competitive Filtering

While a user's preference is directly reflected in the interactive choic...

A session-based song recommendation approach involving user characterization along the play power-law distribution

In recent years, streaming music platforms have become very popular main...

On Sampling Collaborative Filtering Datasets

We study the practical consequences of dataset sampling strategies on th...

Recommending best course of treatment based on similarities of prognostic markers

With the advancement in the technology sector spanning over every field,...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.