Rethinking Neural vs. Matrix-Factorization Collaborative Filtering: the Theoretical Perspectives

by   Da Xu, et al.

The recent work by Rendle et al. (2020), based on empirical observations, argues that matrix-factorization collaborative filtering (MCF) compares favorably to neural collaborative filtering (NCF), and conjectures the dot product's superiority over the feed-forward neural network as similarity function. In this paper, we address the comparison rigorously by answering the following questions: 1. what is the limiting expressivity of each model; 2. under the practical gradient descent, to which solution does each optimization path converge; 3. how would the models generalize under the inductive and transductive learning setting. Our results highlight the similar expressivity for the overparameterized NCF and MCF as kernelized predictors, and reveal the relation between their optimization paths. We further show their different generalization behaviors, where MCF and NCF experience specific tradeoff and comparison in the transductive and inductive collaborative filtering setting. Lastly, by showing a novel generalization result, we reveal the critical role of correcting exposure bias for model evaluation in the inductive setting. Our results explain some of the previously observed conflicts, and we provide synthetic and real-data experiments to shed further insights to this topic.


page 1

page 2

page 3

page 4


Simulated Annealing with Levy Distribution for Fast Matrix Factorization-Based Collaborative Filtering

Matrix factorization is one of the best approaches for collaborative fil...

In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning

We present experiments demonstrating that some other form of capacity co...

Reenvisioning Collaborative Filtering vs Matrix Factorization

Collaborative filtering models based on matrix factorization and learned...

Neural Collaborative Filtering vs. Matrix Factorization Revisited

Embedding based models have been the state of the art in collaborative f...

Predicting clicks in online display advertising with latent features and side-information

We review a method for click-through rate prediction based on the work o...

Predicting Yield Performance of Parents in Plant Breeding: A Neural Collaborative Filtering Approach

Experimental corn hybrids are created in plant breeding programs by cros...

Iterative Collaborative Filtering for Sparse Noisy Tensor Estimation

We consider the task of tensor estimation, i.e. estimating a low-rank 3-...

Please sign up or login with your details

Forgot password? Click here to reset