RecoGym: A Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertising

08/02/2018
by   David Rohde, et al.
0

Recommender Systems are becoming ubiquitous in many settings and take many forms, from product recommendation in e-commerce stores, to query suggestions in search engines, to friend recommendation in social networks. Current research directions which are largely based upon supervised learning from historical data appear to be showing diminishing returns with a lot of practitioners report a discrepancy between improvements in offline metrics for supervised learning and the online performance of the newly proposed models. One possible reason is that we are using the wrong paradigm: when looking at the long-term cycle of collecting historical performance data, creating a new version of the recommendation model, A/B testing it and then rolling it out. We see that there a lot of commonalities with the reinforcement learning (RL) setup, where the agent observes the environment and acts upon it in order to change its state towards better states (states with higher rewards). To this end we introduce RecoGym, an RL environment for recommendation, which is defined by a model of user traffic patterns on e-commerce and the users response to recommendations on the publisher websites. We believe that this is an important step forward for the field of recommendation systems research, that could open up an avenue of collaboration between the recommender systems and reinforcement learning communities and lead to better alignment between offline and online performance metrics.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/15/2022

Rethinking Reinforcement Learning for Recommendation: A Prompt Perspective

Modern recommender systems aim to improve user experience. As reinforcem...
research
05/18/2023

Contrastive State Augmentations for Reinforcement Learning-Based Recommender Systems

Learning reinforcement learning (RL)-based recommenders from historical ...
research
05/01/2019

Beyond Personalization: Research Directions in Multistakeholder Recommendation

Recommender systems are personalized information access applications; th...
research
07/26/2023

Integrating Offline Reinforcement Learning with Transformers for Sequential Recommendation

We consider the problem of sequential recommendation, where the current ...
research
11/07/2020

Do Offline Metrics Predict Online Performance in Recommender Systems?

Recommender systems operate in an inherently dynamical setting. Past rec...
research
12/20/2020

Reinforcement Learning-based Product Delivery Frequency Control

Frequency control is an important problem in modern recommender systems....
research
09/10/2023

Representation Learning in Low-rank Slate-based Recommender Systems

Reinforcement learning (RL) in recommendation systems offers the potenti...

Please sign up or login with your details

Forgot password? Click here to reset