Non-invasive Self-attention for Side Information Fusion in Sequential Recommendation

03/05/2021
by   Chang Liu, et al.
0

Sequential recommender systems aim to model users' evolving interests from their historical behaviors, and hence make customized time-relevant recommendations. Compared with traditional models, deep learning approaches such as CNN and RNN have achieved remarkable advancements in recommendation tasks. Recently, the BERT framework also emerges as a promising method, benefited from its self-attention mechanism in processing sequential data. However, one limitation of the original BERT framework is that it only considers one input source of the natural language tokens. It is still an open question to leverage various types of information under the BERT framework. Nonetheless, it is intuitively appealing to utilize other side information, such as item category or tag, for more comprehensive depictions and better recommendations. In our pilot experiments, we found naive approaches, which directly fuse types of side information into the item embeddings, usually bring very little or even negative effects. Therefore, in this paper, we propose the NOninVasive self-attention mechanism (NOVA) to leverage side information effectively under the BERT framework. NOVA makes use of side information to generate better attention distribution, rather than directly altering the item embedding, which may cause information overwhelming. We validate the NOVA-BERT model on both public and commercial datasets, and our method can stably outperform the state-of-the-art models with negligible computational overheads.

READ FULL TEXT

page 1

page 2

page 3

page 4

08/20/2018

Next Item Recommendation with Self-Attention

In this paper, we propose a novel sequence-aware recommendation model. O...
11/15/2019

Sequential Recommendation with Relation-Aware Kernelized Self-Attention

Recent studies identified that sequential Recommendation is improved by ...
04/25/2022

MLP4Rec: A Pure MLP Architecture for Sequential Recommendations

Self-attention models have achieved state-of-the-art performance in sequ...
01/29/2020

Déjà vu: A Contextualized Temporal Attention Mechanism for Sequential Recommendation

Predicting users' preferences based on their sequential behaviors in his...
04/23/2022

Decoupled Side Information Fusion for Sequential Recommendation

Side information fusion for sequential recommendation (SR) aims to effec...
04/22/2022

Exploiting Session Information in BERT-based Session-aware Sequential Recommendation

In recommendation systems, utilizing the user interaction history as seq...
08/20/2018

Self-Attentive Sequential Recommendation

Sequential dynamics are a key feature of many modern recommender systems...