Self-Supervised Reinforcement Learning forRecommender Systems

06/10/2020
by   Xin Xin, et al.
70

In session-based or sequential recommendation, it is important to consider a number of factors like long-term user engagement, multiple types of user-item interactions such as clicks, purchases etc. The current state-of-the-art supervised approaches fail to model them appropriately. Casting sequential recommendation task as a reinforcement learning (RL) problem is a promising direction. A major component of RL approaches is to train the agent through interactions with the environment. However, it is often problematic to train a recommender in an on-line fashion due to the requirement to expose users to irrelevant recommendations. As a result, learning the policy from logged implicit feedback is of vital importance, which is challenging due to the pure off-policy setting and lack of negative rewards (feedback). In this paper, we propose self-supervised reinforcement learning for sequential recommendation tasks. Our approach augments standard recommendation models with two output layers: one for self-supervised learning and the other for RL. The RL part acts as a regularizer to drive the supervised layer focusing on specific rewards(e.g., recommending items which may lead to purchases rather than clicks) while the self-supervised layer with cross-entropy loss provides strong gradient signals for parameter updates. Based on such an approach, we propose two frameworks namely Self-Supervised Q-learning(SQN) and Self-Supervised Actor-Critic(SAC). We integrate the proposed frameworks with four state-of-the-art recommendation models. Experimental results on two real-world datasets demonstrate the effectiveness of our approach.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/10/2020

Self-Supervised Reinforcement Learning for Recommender Systems

In session-based or sequential recommendation, it is important to consid...
research
11/14/2020

A Geometric Perspective on Self-Supervised Policy Adaptation

One of the most challenging aspects of real-world reinforcement learning...
research
05/10/2022

State Encoders in Reinforcement Learning for Recommendation: A Reproducibility Study

Methods for reinforcement learning for recommendation (RL4Rec) are incre...
research
02/07/2023

Multi-Task Recommendations with Reinforcement Learning

In recent years, Multi-task Learning (MTL) has yielded immense success i...
research
07/31/2021

Sequence Adaptation via Reinforcement Learning in Recommender Systems

Accounting for the fact that users have different sequential patterns, t...
research
11/05/2021

Supervised Advantage Actor-Critic for Recommender Systems

Casting session-based or sequential recommendation as reinforcement lear...
research
10/28/2021

Choosing the Best of Both Worlds: Diverse and Novel Recommendations through Multi-Objective Reinforcement Learning

Since the inception of Recommender Systems (RS), the accuracy of the rec...

Please sign up or login with your details

Forgot password? Click here to reset