Modeling Sequences as Distributions with Uncertainty for Sequential Recommendation

by   Ziwei Fan, et al.

The sequential patterns within the user interactions are pivotal for representing the user's preference and capturing latent relationships among items. The recent advancements of sequence modeling by Transformers advocate the community to devise more effective encoders for the sequential recommendation. Most existing sequential methods assume users are deterministic. However, item-item transitions might fluctuate significantly in several item aspects and exhibit randomness of user interests. This stochastic characteristics brings up a solid demand to include uncertainties in representing sequences and items. Additionally, modeling sequences and items with uncertainties expands users' and items' interaction spaces, thus further alleviating cold-start problems. In this work, we propose a Distribution-based Transformer for Sequential Recommendation (DT4SR), which injects uncertainties into sequential modeling. We use Elliptical Gaussian distributions to describe items and sequences with uncertainty. We describe the uncertainty in items and sequences as Elliptical Gaussian distribution. And we adopt Wasserstein distance to measure the similarity between distributions. We devise two novel Trans-formers for modeling mean and covariance, which guarantees the positive-definite property of distributions. The proposed method significantly outperforms the state-of-the-art methods. The experiments on three benchmark datasets also demonstrate its effectiveness in alleviating cold-start issues. The code is available in


page 1

page 2

page 3

page 4


Sequential Recommendation via Stochastic Self-Attention

Sequential recommendation models the dynamics of a user's previous behav...

Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer

Sequential Recommendation characterizes the evolving patterns by modelin...

Dynamic Graph Neural Networks for Sequential Recommendation

Modeling users' preference from his historical sequences is one of the c...

Regularizing Matrix Factorization with User and Item Embeddings for Recommendation

Following recent successes in exploiting both latent factor and word emb...

A novel HD Computing Algebra: Non-associative superposition of states creating sparse bundles representing order information

Information inflow into a computational system is by a sequence of infor...

Feature-based factorized Bilinear Similarity Model for Cold-Start Top-n Item Recommendation

Recommending new items to existing users has remained a challenging prob...

M2pht: Mixed Models with Preferences and Hybrid Transitions for Next-Basket Recommendation

Next-basket recommendation considers the problem of recommending a set o...