EvalRS: a Rounded Evaluation of Recommender Systems

07/12/2022
by   Jacopo Tagliabue, et al.
0

Much of the complexity of Recommender Systems (RSs) comes from the fact that they are used as part of more complex applications and affect user experience through a varied range of user interfaces. However, research focused almost exclusively on the ability of RSs to produce accurate item rankings while giving little attention to the evaluation of RS behavior in real-world scenarios. Such narrow focus has limited the capacity of RSs to have a lasting impact in the real world and makes them vulnerable to undesired behavior, such as reinforcing data biases. We propose EvalRS as a new type of challenge, in order to foster this discussion among practitioners and build in the open new methodologies for testing RSs "in the wild".

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/14/2023

EvalRS 2023. Well-Rounded Recommender Systems For Real-World Deployments

EvalRS aims to bring together practitioners from industry and academia t...
research
09/11/2019

RecSim: A Configurable Simulation Platform for Recommender Systems

We propose RecSim, a configurable platform for authoring simulation envi...
research
07/19/2022

Group Validation in Recommender Systems: Framework for Multi-layer Performance Evaluation

Interpreting the performance results of models that attempt to realize u...
research
08/21/2018

Who is Really Affected by Fraudulent Reviews? An analysis of shilling attacks on recommender systems in real-world scenarios

We present the results of an initial analysis conducted on a real-life s...
research
05/30/2021

Personalization in E-Grocery: Top-N versus Top-k Rankings

Business success in e-commerce depends on customer perceived value. A cu...
research
09/13/2019

Towards Sharing Task Environments to Support Reproducible Evaluations of Interactive Recommender Systems

Beyond sharing datasets or simulations, we believe the Recommender Syste...
research
07/21/2023

Methodologies for Improving Modern Industrial Recommender Systems

Recommender system (RS) is an established technology with successful app...

Please sign up or login with your details

Forgot password? Click here to reset