Initialization Matters: Regularizing Manifold-informed Initialization for Neural Recommendation Systems

by   Yinan Zhang, et al.

Proper initialization is crucial to the optimization and the generalization of neural networks. However, most existing neural recommendation systems initialize the user and item embeddings randomly. In this work, we propose a new initialization scheme for user and item embeddings called Laplacian Eigenmaps with Popularity-based Regularization for Isolated Data (LEPORID). LEPORID endows the embeddings with information regarding multi-scale neighborhood structures on the data manifold and performs adaptive regularization to compensate for high embedding variance on the tail of the data distribution. Exploiting matrix sparsity, LEPORID embeddings can be computed efficiently. We evaluate LEPORID in a wide range of neural recommendation models. In contrast to the recent surprising finding that the simple K-nearest-neighbor (KNN) method often outperforms neural recommendation systems, we show that existing neural systems initialized with LEPORID often perform on par or better than KNN. To maximize the effects of the initialization, we propose the Dual-Loss Residual Recommendation (DLR2) network, which, when initialized with LEPORID, substantially outperforms both traditional and state-of-the-art neural recommender systems.


page 1

page 2

page 3

page 4


Convolutional Gaussian Embeddings for Personalized Recommendation with Uncertainty

Most of existing embedding based recommendation models use embeddings (v...

Capturing Popularity Trends: A Simplistic Non-Personalized Approach for Enhanced Item Recommendation

Recommender systems have been gaining increasing research attention over...

Federated Recommendation with Additive Personalization

With rising concerns about privacy, developing recommendation systems in...

Learning Consumer and Producer Embeddings for User-Generated Content Recommendation

User-Generated Content (UGC) is at the core of web applications where us...

Where are we in embedding spaces? A Comprehensive Analysis on Network Embedding Approaches for Recommender Systems

Hyperbolic space and hyperbolic embeddings are becoming a popular resear...

Multi-Scale Quasi-RNN for Next Item Recommendation

How to better utilize sequential information has been extensively studie...

Please sign up or login with your details

Forgot password? Click here to reset