A novel Empirical Bayes with Reversible Jump Markov Chain in User-Movie Recommendation system

08/15/2018 ∙ by Arabin Kumar Dey, et al. ∙ 0

In this article we select the unknown dimension of the feature by re- versible jump MCMC inside a simulated annealing in bayesian set up of collaborative filter. We implement the same in MovieLens small dataset. We also tune the hyper parameter by using a modified empirical bayes. It can also be used to guess an initial choice for hyper-parameters in grid search procedure even for the datasets where MCMC oscillates around the true value or takes long time to converge.



There are no comments yet.


page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Matrix factorization is a popular technique in collaborative filtering to predict the missing ratings in user-movie recommendation system. In a very recent paper of Dey et al. Dey et al. (2017)

, bayesian formulation of this problem is discussed. Hyper prameter choice was the major issue in that paper. However this problem was attempted only after suitable choice of feature dimension for user and movie feature vector. Usual method used to select such feature dimension is nothing but one dimensional grid search that select the dimension which minimizes the test error. This approach is boring as it attracts extra computational burden to select the feature dimension. In this paper we adapt a bayesian paradigm which automatically select the feature dimension along with hyper parameters which minimizes the cost function. We propose to use simulated annealing instead of straight forward metropolis hastings so that we can extract the global optimal value of the feature vector. The idea of using simulated annealing in collaborative filter is recently attempted by few researchers [

Feng and Su (2013), Shehata et al. (2017), Picot-Clemente et al. (2010)]. However selecting the feature dimension automatically in user-movie recommendation set up is not attempted by anyone of the author. We propose to use reversible jump MCMC in simulated annealing set up. However such optimization still requires grid search on hyper parameters. So we propose to use empirical bayes used by Atchade to take care the choice of hyper parameters. But this does not fully solve the implementation problem as such algorithm may depend on initial choice of the hyper-parameters. So we modify the empirical bayes further and we call it an Adam-based empirical algorithm.

Bayesian model is used in many papers related to collaborative filter (Chien and George (1999), Jin and Si (2004), Xiong et al. (2010), Yu et al. (2002)). Ansari et al. (2000) propose a bayesian preference model that statistically involves several types of information useful for making recommendations, such as user preferences, user and item features and expert evaluations. A fully bayesian treatment of the Probabilistic Matrix Factorization (PMF) model is discussed by Salakhutdinov and Mnih (2008)

in which model capacity is controlled automatically under Gibbs sampler integrating over all model parameters and hyperparameters. One of the important objective of Collaborative Filter (

Su and Khoshgoftaar (2009), N. Liu et al. (2009)) is to handle the highly sparse data. Our proposed automation in the algorithm allow us not only to take care such problems, but it also makes the algorithm more scalable.

The rest of the paper is orgnized as follows. In section 2, we provide usual collaborative filter algorithm and address few problems. Implementation of simulated annealing along with Reversible Jump MCMC and Empirical Bayes method is used in section 3. Results on user-movie data set is provided in Section 4. We conclude the paper in section 5.

2 Learning latent features

Definition 2.1.

:= Number of unique users,
:= Number of unique items,
:= Number of latent feature,
:= Sparse rating matrix of dimension where the element of the rating is given by user to item .
:= The user feature matrix of dimension where row represents the user feature vector .
:= The item feature matrix of dimension where row represents the item feature vector .

:= The loss function which is to be minimized.

& := User and item hyper-parameters in the regularized Loss function .
:= Frobenius norm for matrices.
:= The set of user-item indices in the sparse rating matrix for which the ratings are known.
:= The set of indices of item indices for user for which the ratings are known.
:= The set of indices of user indices for item who rated it.

When latent feature dimension is known, the system minimizes the regularized loss on the set of known ratings to find optimal set of parameter vectors and , for all and .


This Loss function is a biconvex function in and and can be iteratively optimized by regularized least square methods keeping the hyper-parameters fixed.

The user and item feature matrices are first heuristically initialized using normal random matrices with iid. entries such that the product

has a mean of 3 and variance 1.

The iteration is broken into 2 steps until test loss convergence. The first step computes the regularized least squares estimate for each of the user feature vectors

from their known ratings.
The second step computes the regularized least squares estimate for each of the item feature vectors from their known ratings.

The first step minimizes the below expression keeping fixed.


The second step minimizes the below expression keeping fixed.


The normal equations corresponding to the regularized least squares solution for user feature vectors are


The normal equations corresponding to the regularized least squares solution for item feature vectors are


Iteratively minimizing the loss function gives us a local minima (possibly global).

2.1 Drawbacks of the previous method

One of the major drawback of the previous method is finding the choice of and . Usual method uses grid search method. Moreover choice of feature dimension makes the grid search approach computationally less efficient.

3 Proposed Method

We propose to use simulated annealing Welsh (1989) in this set up which connects this optimization set in bayesian paradigm. Reseacher has already used this optimization in some recent work. However capturing the dimension simultaneously in simulated annealing is not available. We use reversible jump MCMC inside metropolis hastings steps of simulated annealing.

3.1 Ordinary Simulated Annealing in User-movie set up

Given the loss function in equation-1, we get the corresponding Boltzmann Distribution as posterior distribution of given ,


where is the temperature and each component of prior vectors or user and movie feature vectors are independent and identically distributed normal variates. Variance of each of the user movie parameters are and . Clearly and are hyper parameters.

3.2 Simulated Annealing with Reversible Jump MCMC in Empirical Bayes Set up

Reversible Jump MCMC is used when dimension in the given problem itself is an unknown quantity. In this context, user and movie feature dimension is unknown. We assume maximum possible dimension as 50. At each iteration of MCMC, we choose a dimension uniformly from the set . Temparature is set in simulated annealing as . At each move, new point is chosen based on old point via an orthogonal Gram-Chalier transformation . If and , then Gram-Chalier orthogonal transformation uses the following relation :

Since, choice of hyperparameter and is crucial, we use empirical bayes to select them based on the data set. In one of the case-study in user-movie set up, Dey et al. Dey et al. (2017) used empirical bayes proposed by Atchade Atchadé (2011). The algorithm is as follows :

The Stochastic approximation algorithm proposed to maximize the function of hyperparameter given the observations () as :

  • Generate .

  • Calculate

where, is the parameter vector generated from transition kernel given the hyper-parameter where is such that


Note that is the posterior function of the parameter given the data y.

First step of stochastic approximation algorithm is to draw sample from transition kernel which is our proposed reversible jump markov chain with simulated annealing algorithm to obtain a sequence of random samples from . Let and be the current iterates of the iteration sequence and be the proposal distribution. The algorithmic steps to get the next iterate is

  • Set : Initial choice of

  • ,

  • while ,

    • .

    • Unif(0, 1).

    • if()

    • Generate Unif(0, 1) of dimension , which contains numbers from and numbers from .

    • ; where is a Gram-chalier orthogonal transformation such that

    • Calculate

      where move probability

    • if()

    • accept and update the hyperparameters using empirical bayes.

    • else accept x and update hyper-parameters using empirical bayes.

  • if()

  • Make a birth move from to . We should get so that , where is the inverse map of .

  • where, and

  • if()

  • accept and update the hyperparameters using empirical bayes.

  • else accept x and update hyper-parameters using empirical bayes.

  • if()

  • Choose from dimension from .

  • Apply usual simulated annealing and update the hyper parameters as similar to previous one.

  • At each iteration decrease the temparature.

According to the algorithm, we sample one set of and using Metropolis-Hasting algorithm and use them to update hyper-parameters in the next step.




From the definition of in Equation-7, we have




In this case the sequence is chosen adaptively based on a well-known algorithm called Adam.

Adam needs the following steps of tuning the hyper parameters in context of above empirical bayes set up.

  • The following hyper parameters are required : = step size;

    Exponential decay rates for moment estimates.

    : stochastic objective function with parameter . Initial choice of parameter vector.

  • Assume as initial choice of parameter vector. Initialize , and .

  • Repeat the following steps until convergence.

    • [Update biased first moment e stimate]

    • [Update biased first moment estimate]

    • [Update biased first moment estimate]

    • [Update second raw moment estimate]

    • ; ; [Compute bias-corrected 1st moment] , [Compute bias-corrected 2nd moment]

    • ;

  • Return , .

4 Data Analysis and Results

The benchmark datasets used in the experiments were the 3 Movie Lens dataset. They can be found at http://grouplens.org/datasets/movielens/. We implement the above algorithm in old MovieLens small dataset which has been used with 943 users and 1682 movie. We take 80% data as training and 20% data as test set. Initial choice of hyparameters is set to and , , , , . The vectors and were initialized with iid standard normal random numbers. We stop the updates of Hyper-Parameters and if changes in succesive two hyper-parameters falls below tolerance, where tolerance is set to .

Test and Train smoothed Loss is drawn based on a fixed size window of 10 iterations in Figure-1. Minimum Training Cost can be found at k=2 and Test RMSE at this point as: 1.02089331071. However the test loss stabilizes at k = 4. Converged Hyper-Parameters: 19.439, 25.796.

Figure 1: Test and Train smoothed Loss over different iteration

5 Conclusion

The paper describes successful implementation of choice of hidden feature dimension along with automatically tuning all hyper-parameters based on the data set in a collaborative filter set up. The result depends on the selection of test and trainng set. Transformation matrix calculation makes the algorithm little slow. We can modify the algorithm taking levy-type proposal density in this automated simulated annealing. There some variation of the model can proposed in this direction. The work is on progress.


  • (1)
  • Ansari et al. (2000) Asim Ansari, Skander Essegaier, and Rajeev Kohli. 2000. Internet recommendation systems. (2000).
  • Atchadé (2011) Yves F Atchadé. 2011. A computational framework for empirical Bayes inference. Statistics and computing 21, 4 (2011), 463–473.
  • Chien and George (1999) Yung-Hsin Chien and Edward I George. 1999. A bayesian model for collaborative filtering. In AISTATS.
  • Dey et al. (2017) Arabin Kumar Dey, Raghav Somani, and Sreangsu Acharyya. 2017. A case study of empirical Bayes in a user-movie recommendation system. Communication in Statistics : Case Studies, Data Analysis and Applications 3, 1 -2 (2017), 1–6.
  • Feng and Su (2013) Zhi Ming Feng and Yi Dan Su. 2013. Application of Using Simulated Annealing to Combine Clustering with Collaborative Filtering for Item Recommendation. In Applied Mechanics and Materials, Vol. 347. Trans Tech Publ, 2747–2751.
  • Jin and Si (2004) Rong Jin and Luo Si. 2004.

    A Bayesian approach toward active learning for collaborative filtering. In

    Proceedings of the 20th conference on Uncertainty in artificial intelligence

    . AUAI Press, 278–285.
  • N. Liu et al. (2009) Nathan N. Liu, Min Zhao, and Qiang Yang. 2009. Probabilistic latent Preference Analysis for Collaborative Filtering. In Proceedings of the 18th ACM conference on information and knowledge management. ACM, 759–766.
  • Picot-Clemente et al. (2010) Romain Picot-Clemente, Christophe Cruz, and Christophe Nicolle. 2010. A Semantic-based Recommender System Using A Simulated Annealing Algorithm. In SEMAPRO 2010, The Fourth international Conference on Advances in Semantic Processing. ISBN–978.
  • Salakhutdinov and Mnih (2008) Ruslan Salakhutdinov and Andriy Mnih. 2008. Bayesian probabilistic matrix factorization using Markov chain Monte Carlo. In

    Proceedings of the 25th international conference on Machine learning

    . ACM, 880–887.
  • Shehata et al. (2017) Mostafa A Shehata, Mohammad Nassef, and Amr A Badr. 2017. Simulated Annealing with Levy Distribution for Fast Matrix Factorization-Based Collaborative Filtering. arXiv preprint arXiv:1708.02867 (2017).
  • Su and Khoshgoftaar (2009) Xiaoyuan Su and Taghi M Khoshgoftaar. 2009. A survey of collaborative filtering techniques. Advances in artificial intelligence 2009 (2009), 4.
  • Xiong et al. (2010) Liang Xiong, Xi Chen, Tzu-Kuo Huang, Jeff Schneider, and Jaime G Carbonell. 2010.

    Temporal collaborative filtering with bayesian probabilistic tensor factorization. In

    Proceedings of the 2010 SIAM International Conference on Data Mining. SIAM, 211–222.
  • Yu et al. (2002) Kai Yu, Anton Schwaighoferi, and Volker Tresp. 2002. Collaborative ensemble learning : Combining collaborative and content based information filtering via hierarchical Bayes. In Proceedings of Nineteenth Conference on Uncertainty in Artificial Intelligence. Morgan Kaufmann Publishers Inc., 616–623.