Non-invasive Self-attention for Side Information Fusion in Sequential Recommendation

by   Chang Liu, et al.

Sequential recommender systems aim to model users' evolving interests from their historical behaviors, and hence make customized time-relevant recommendations. Compared with traditional models, deep learning approaches such as CNN and RNN have achieved remarkable advancements in recommendation tasks. Recently, the BERT framework also emerges as a promising method, benefited from its self-attention mechanism in processing sequential data. However, one limitation of the original BERT framework is that it only considers one input source of the natural language tokens. It is still an open question to leverage various types of information under the BERT framework. Nonetheless, it is intuitively appealing to utilize other side information, such as item category or tag, for more comprehensive depictions and better recommendations. In our pilot experiments, we found naive approaches, which directly fuse types of side information into the item embeddings, usually bring very little or even negative effects. Therefore, in this paper, we propose the NOninVasive self-attention mechanism (NOVA) to leverage side information effectively under the BERT framework. NOVA makes use of side information to generate better attention distribution, rather than directly altering the item embedding, which may cause information overwhelming. We validate the NOVA-BERT model on both public and commercial datasets, and our method can stably outperform the state-of-the-art models with negligible computational overheads.


page 1

page 2

page 3

page 4


Next Item Recommendation with Self-Attention

In this paper, we propose a novel sequence-aware recommendation model. O...

Sequential Recommendation with Relation-Aware Kernelized Self-Attention

Recent studies identified that sequential Recommendation is improved by ...

MLP4Rec: A Pure MLP Architecture for Sequential Recommendations

Self-attention models have achieved state-of-the-art performance in sequ...

Déjà vu: A Contextualized Temporal Attention Mechanism for Sequential Recommendation

Predicting users' preferences based on their sequential behaviors in his...

Decoupled Side Information Fusion for Sequential Recommendation

Side information fusion for sequential recommendation (SR) aims to effec...

Exploiting Session Information in BERT-based Session-aware Sequential Recommendation

In recommendation systems, utilizing the user interaction history as seq...

Self-Attentive Sequential Recommendation

Sequential dynamics are a key feature of many modern recommender systems...