DeepRec: An Open-source Toolkit for Deep Learning based Recommendation

1 Introduction

In recent years, deep learning has achieved tremendous success in many application domains, such as computer vision, natural language processing, and speech recognition. Inspired by the effectiveness of deep neural networks, researchers and practitioners started to apply deep learning techniques to recommendation tasks and have achieved a huge improvement on many recommendation tasks. Meanwhile, the number of publications on deep neural network based recommendation models also increases exponentially

[Zhang et al.2019]. Moreover, many companies resort to deep learning techniques to improve their recommendation quality for better user experience and increasing revenue [Covington et al.2016, Cheng et al.2016]. In general, deep learning techniques can be utilized to learn representations for user and item side information (e.g., user profile, item content, etc.), introduce nonlinearities to recommendation models and model the sequential patterns of user consumption history. In addition, deep neural networks are of high flexibility, which makes it possible to develop more powerful models by combining different neural networks.

However, this hype also poses a big challenge to researchers and practitioners, that it, reproducing results for those proposed deep learning based models. Although some authors provide source code for reproductivity, many of them are written in different programming languages, or use different deep learning packages, let alone a large portion of work do not provide source code, which makes it hard to understand and hinder the process of justifying newly proposed methods. To this end, we start the project - DeepRec. Our goal is to make reproducing results easier and lower the bar of developing deep learning based recommendation models and increase their practicality.

Although there are some open source recommendation packages available, such as MyMediaLite²²2http://mymedialite.net/index.html, LibRec [Guo et al.2015], Suprise³³3http://surpriselib.com/ and OpenRec [Yang et al.2018], most of them only include traditional recommendation algorithms or have not updated for a long time. The OpenRec library do provide several deep neural network based recommendation models, the number is still very small (only four models implemented) compared with DeepRec.

To this end, we develop an open-source toolkit named ‘DeepRec’. This toolkit provides a flexible framework and unified interfaces for deep learning based recommendation models. The prime features of DeepRec are twofold:

In this toolkit, we addressed three common scenarios in recommendation research: rating prediction, top-n recommendation (item ranking) and sequence-aware recommendation. It offers state-of-the-art algorithms for those three tasks.
DeepRec is extensible, adaptable and efficient. Users could add algorithm easily. Moreover, DeepRec is able to make use of the computation power of GPUs, making it scalable to large-scale datasets.

2 Introduction to DeepRec

In this section, we briefly introduce the archecture of the DeepRec, plus implemented recommendation algorithms and comparison with other frameworks.

2.1 Architecture of DeepRec

DeepRec is made up of three components: recommender, backend and utility. We adopt the widely used deep learning framework Tensorflow and several other frequent used scientific calculation libraries such as Numpy, Scipy, Sklearn and Pandas as the core backend; A number of representative and advanced algorithms deep learning based recommendation models forms the recommender components. We split the algorithms into three categories: rating prediction, top-n recommendation and sequence-aware recommendation. We put the common evaluators and data preprocess functions into the utility component.

Figure 1: Architecture of DeepRec. It consists of three important components: recommender, backend and utility.

2.2 Implemented Algorithms

A number of algorithms that were implemented in the current version are listed below. Note that a few non deep learning methods (e.g., biasedSVD, Factorization Machines, BPRMF) are also implemented with Tensorflow.

2.2.1 Rating Prediction Models

In this category, we implemented methods like classical method like biasedSVD (or MF) [Koren et al.2009], Factorization Machines (FM) [Rendle2010], user and item based AutoRec [Sedhain et al.2015], neural network matrix factorization (NNMF) [Dziugaite and Roy2015], neural rating regression (NRR) [Li et al.2017], neural factorization machines (NFM) [He and Chua2017] and attentive factorization machines (AFM) [Xiao et al.2017]. For factorization machines, we use the same feature format as libfm [Rendle2012] to make them compatible.

2.2.2 Top-n Recommendation Models

Here, we included classical pairwise personalized ranking methods BPRMF [Rendle et al.2009], collaborative metric learning (CML) [Hsieh et al.2017], collaborative denoising auto-Encoder (CDAE) [Wu et al.2016]

, multilayer-perceptron (MLP), generalized matrix factorization (GMF) and neural collaborative filtering (NeuMF)

[he2017neural], NeuRec [Zhang et al.2018b] and latent relational metric learning [Tay et al.2018].

2.2.3 Sequential Recommendation Models

For sequence-aware recommendation models, we implemented the personalized ranking metric embedding model (PRME) [Feng et al.2015], LSTM and GRU based methods [Hidasi et al.2015], convolution based model (Caser) [Tang and Wang2018], and self-attention based approach (AttRec) [Zhang et al.2018a].

2.3 Evaluation Metrics

Several commonly used evaluation metrics were included in this toolkit. For rating prediction, we can use Root Mean Square Error (RMSE) and Mean Average Error (MAE) to measure the effectiveness. For top-n recommendation and sequence-aware recommendation, we included several ranking-aware metrics, including, Recall@n, Precision@n, Normalized Discounted Cumulative Gain (NDCG) and Mean Reciprocal Rank (MRR) to measure both the accuracy and quality of ranking lists. For detailed definitions, readers are referred to

[Shani and Gunawardana2011].

2.4 Comparison with Other Frameworks

Library	backend	GPUs	3 scenarios	#DL
Suprise	SciKits	N	N	0
MyMediaLite	N/A	N	N	0
LibRec	N/A	N	N	0
OpenRec	Tensorflow	Y	N	4
DeepRec	Tensorflow	Y	Y	> 20

Table 1: Comparison with existing libraries. “3 scenarios” means whether the library can deal with the above mentioned three different tasks. “#DL” means the number of neural networks based methods.

Table 1 summarizes the difference between our toolkit and other existing Toolkits. Suprise, MyMedialite, LibRec do not include neural network based models and can not utilize the computation power of GPUs. OpenRec also adopts Tensorflow as the backend, but it only implemented four ranking based recommendation methods, while DeepRec includes over twenty state-of-the-art deep learning based recommendation model. In addition, DeepRec is the first library that includes sequence-aware models.

3 Conclusion

In this demo paper, we introduce the open-source library DeepRec, in the hope that it could ease the burden of researchers and practitioners on reproducing existing models. DeepRec is a flexible, extensible and efficient framework which includes a number of state-of-the-art deep learning based recommendation algorithms.

Moving forward, future work will include: implementing more models and evaluation measures, improving the documentation and tutorials as well as updating it to follow the scheme of Tensorflow 2.0.

References

[Cheng et al.2016] Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. Wide & deep learning for recommender systems. In Proceedings of the 1st workshop on deep learning for recommender systems, pages 7–10. ACM, 2016.
[Covington et al.2016] Paul Covington, Jay Adams, and Emre Sargin. Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM conference on recommender systems, pages 191–198. ACM, 2016.
[Dziugaite and Roy2015] Gintare Karolina Dziugaite and Daniel M. Roy. Neural network matrix factorization. CoRR, abs/1511.06443, 2015.
[Feng et al.2015] Shanshan Feng, Xutao Li, Yifeng Zeng, Gao Cong, Yeow Meng Chee, and Quan Yuan. Personalized ranking metric embedding for next new poi recommendation. In
Twenty-Fourth International Joint Conference on Artificial Intelligence
, 2015.
[Guo et al.2015] Guibing Guo, Jie Zhang, Zhu Sun, and Neil Yorke-Smith. Librec: A java library for recommender systems. In UMAP Workshops, volume 4, 2015.
[He and Chua2017] Xiangnan He and Tat-Seng Chua. Neural factorization machines for sparse predictive analytics. In Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval, pages 355–364. ACM, 2017.
[Hidasi et al.2015] Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939, 2015.
[Hsieh et al.2017] Cheng-Kang Hsieh, Longqi Yang, Yin Cui, Tsung-Yi Lin, Serge Belongie, and Deborah Estrin. Collaborative metric learning. In Proceedings of the 26th international conference on world wide web, pages 193–201. International World Wide Web Conferences Steering Committee, 2017.
[Koren et al.2009] Yehuda Koren, Robert Bell, and Chris Volinsky. Matrix factorization techniques for recommender systems. Computer, (8):30–37, 2009.
[Li et al.2017] Piji Li, Zihao Wang, Zhaochun Ren, Lidong Bing, and Wai Lam. Neural rating regression with abstractive tips generation for recommendation. In Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval, pages 345–354. ACM, 2017.
[Rendle et al.2009] Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. Bpr: Bayesian personalized ranking from implicit feedback. In UAI, 2009.
[Rendle2010] Steffen Rendle. Factorization machines. In 2010 IEEE International Conference on Data Mining, pages 995–1000. IEEE, 2010.
[Rendle2012] Steffen Rendle. Factorization machines with libfm. ACM Transactions on Intelligent Systems and Technology (TIST), 3(3):57, 2012.
[Sedhain et al.2015] Suvash Sedhain, Aditya Krishna Menon, Scott Sanner, and Lexing Xie.
Autorec: Autoencoders meet collaborative filtering.
In WWW, 2015.
[Shani and Gunawardana2011] Guy Shani and Asela Gunawardana. Evaluating recommendation systems. In Recommender systems handbook, pages 257–297. Springer, 2011.
[Tang and Wang2018] Jiaxi Tang and Ke Wang. Personalized top-n sequential recommendation via convolutional sequence embedding. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, pages 565–573. ACM, 2018.
[Tay et al.2018] Yi Tay, Luu Anh Tuan, and Siu Cheung Hui. Latent relational metric learning via memory-based attention for collaborative ranking. In Proceedings of the 2018 World Wide Web Conference on World Wide Web, pages 729–739. International World Wide Web Conferences Steering Committee, 2018.
[Wu et al.2016] Yao Wu, Christopher DuBois, Alice X Zheng, and Martin Ester. Collaborative denoising auto-encoders for top-n recommender systems. In Proceedings of the Ninth ACM International Conference on Web Search and Data Mining, pages 153–162. ACM, 2016.
[Xiao et al.2017] Jun Xiao, Hao Ye, Xiangnan He, Hanwang Zhang, Fei Wu, and Tat-Seng Chua. Attentional factorization machines: Learning the weight of feature interactions via attention networks. arXiv preprint arXiv:1708.04617, 2017.
[Yang et al.2018] Longqi Yang, Eugene Bagdasaryan, Joshua Gruenstein, Cheng-Kang Hsieh, and Deborah Estrin. Openrec: A modular framework for extensible and adaptable recommendation algorithms. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, pages 664–672. ACM, 2018.
[Zhang et al.2018a] Shuai Zhang, Yi Tay, Lina Yao, and Aixin Sun. Next item recommendation with self-attention. arXiv preprint arXiv:1808.06414, 2018.
[Zhang et al.2018b] Shuai Zhang, Lina Yao, Aixin Sun, Sen Wang, Guodong Long, and Manqing Dong. Neurec: on nonlinear transformation for personalized ranking. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, pages 3669–3675. AAAI Press, 2018.
[Zhang et al.2019] Shuai Zhang, Lina Yao, Aixin Sun, and Yi Tay. Deep learning based recommender system: A survey and new perspectives. ACM Comput. Surv., 52(1):5:1–5:38, February 2019.

DeepRec: An Open-source Toolkit for Deep Learning based Recommendation

Authors

CytonRL: an Efficient Reinforcement Learning Open-source Toolkit Implemented in C++

CRSLab: An Open-Source Toolkit for Building Conversational Recommender System

DGL-LifeSci: An Open-Source Toolkit for Deep Learning on Graphs in Life Science

MMFashion: An Open-Source Toolbox for Visual Fashion Analysis

Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches

DeepKE: A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Population

AutoRec: An Automated Recommender System

Code Repositories

DeepRec

1 Introduction

2 Introduction to DeepRec

2.1 Architecture of DeepRec

2.2 Implemented Algorithms

2.2.1 Rating Prediction Models

2.2.2 Top-n Recommendation Models

2.2.3 Sequential Recommendation Models

2.3 Evaluation Metrics

2.4 Comparison with Other Frameworks

3 Conclusion

References

DeepRec: An Open-source Toolkit for Deep Learning based Recommendation

Authors

Related Research

CytonRL: an Efficient Reinforcement Learning Open-source Toolkit Implemented in C++

CRSLab: An Open-Source Toolkit for Building Conversational Recommender System

DGL-LifeSci: An Open-Source Toolkit for Deep Learning on Graphs in Life Science

MMFashion: An Open-Source Toolbox for Visual Fashion Analysis

Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches

DeepKE: A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Population

AutoRec: An Automated Recommender System

Code Repositories

DeepRec

This week in AI

1 Introduction

2 Introduction to DeepRec

2.1 Architecture of DeepRec

2.2 Implemented Algorithms

2.2.1 Rating Prediction Models

2.2.2 Top-n Recommendation Models

2.2.3 Sequential Recommendation Models

2.3 Evaluation Metrics

2.4 Comparison with Other Frameworks

3 Conclusion

References