Recommender May Not Favor Loyal Users

04/12/2022
by   Yitong Ji, et al.
0

In academic research, recommender systems are often evaluated on benchmark datasets, without much consideration about the global timeline. Hence, we are unable to answer questions like: Do loyal users enjoy better recommendations than non-loyal users? Loyalty can be defined by the time period a user has been active in a recommender system, or by the number of historical interactions a user has. In this paper, we offer a comprehensive analysis of recommendation results along global timeline. We conduct experiments with five widely used models, i.e., BPR, NeuMF, LightGCN, SASRec and TiSASRec, on four benchmark datasets, i.e., MovieLens-25M, Yelp, Amazon-music, and Amazon-electronic. Our experiment results give an answer "No" to the above question. Users with many historical interactions suffer from relatively poorer recommendations. Users who stay with the system for a short time period enjoy better recommendations. Both findings are counter-intuitive. Interestingly, users who have recently interacted with the system, with respect to the time point of the test instance, enjoy better recommendations. The finding on recency applies to all users, regardless of users' loyalty. Our study offers a different perspective to understand recommender performance, and our findings could trigger a revisit of recommender model design.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/09/2022

From Counter-intuitive Observations to a Fresh Look at Recommender System

Recently, a few papers report counter-intuitive observations made from e...
research
08/01/2020

An Empirical Study of Clarifying Question-Based Systems

Search and recommender systems that take the initiative to ask clarifyin...
research
12/14/2022

How Researchers Could Obtain Quick and Cheap User Feedback on their Algorithms Without Having to Operate their Own Recommender System

The majority of recommendation algorithms are evaluated on the basis of ...
research
04/15/2023

More Is Less: When Do Recommenders Underperform for Data-rich Users?

Users of recommender systems tend to differ in their level of interactio...
research
01/10/2023

Why People Skip Music? On Predicting Music Skips using Deep Reinforcement Learning

Music recommender systems are an integral part of our daily life. Recent...
research
05/27/2020

How to Retrain Recommender System? A Sequential Meta-Learning Method

Practical recommender systems need be periodically retrained to refresh ...
research
06/04/2020

Digital interfaces of historical newspapers: opportunities, restrictions and recommendations

Many libraries offer free access to digitised historical newspapers via ...

Please sign up or login with your details

Forgot password? Click here to reset