1. Introduction
The effectiveness of recommender systems (RS) often relies on how well users’ interests or preferences can be understood and interactions between users and items can be modeled. Collaborative Filtering (CF) (Koren et al., 2009) is one of the widely used and prominent techniques for RS. The underlying assumption of the CF approach is that if a user shares a common item with another user , is also likely to be interested in other items liked by . Although CF has been successfully applied to many recommendation applications, the coldstart problem is considered as one of its major challenges (Koren et al., 2009). The problem arises when a user interacted with a very small number of items. Consequently, the user shares few items with other users, and effectively recommending for the user becomes a challenging task for RS.
If we formulate the relationships between users and items as a bipartite graph^{1}^{1}1
In this paper, we use the terminology ”graph” to refer to the graph/network structure of data and ”network” for the architecture of machine learning models.
, we argue that the connectivity information of the graph can play an important role for tackling the coldstart problem. For example, let us see a bipartite graph in Figure 1. A coldstart user only interacts with item . Since shares with user and user , as a result, three items (, and ) connected with or can all be recommended to by a CFbased model. However, a natural and important question arises: which one in the three items is the most reliable recommendation for ? The key to answer the question lies in the useritem connectivity information. In fact, if we take a look at the connections of the graph, it is clear that there is only one path existing between and (or ), while two paths connect to . Thus, compared with and , obviously, is a more reliable recommendation for .However, existing CFbased methods, including modelbased and memorybased approaches, often suffer from the difficulty of modeling the connectivity information. Previous modelbased approaches, such as Matrix Factorization (MF) (Koren et al., 2009), are usually designed to approximate the direct connections (or proximities). However, indirect connectivity information hidden in the graph structures is rarely captured by traditional modelbased approaches. For instance, it is formidable for them to model the number of paths between and in Figure 1. Whereas a number of memorybased approaches (Sarwar et al., 2001; Jamali and Ester, 2009) is introduced to model the connectivity information, these methods often rely on predefined similarity functions. However, in the real world, defining an appropriate similarity function suitable for diverse application cases is never an easy task.
Spectral graph theory (Shuman et al., 2013)
studies connections between combinatorial properties of a graph and the eigenvalues of matrices associated to the graph, such as the laplacian matrix (see Definition
2.4 in Section 2). In general, the spectrum of a graph focuses on the connectivity of the graph, instead of the geometrical proximity. To see how does the spectral domain come to help for recommendations and better understand the advantages of viewing a useritem bipartite graph in the spectral perspective, let us revisit the toy example shown in Figure 1. For the bipartite graph, we visually plot its vertices in one specific frequency domain. Although vertices do not come with coordinates, a popular way to draw them in a space is to use eigenvectors of a laplacian matrix associated with the graph to supply coordinates
(Spielman, 2007). Figure 2 shows that, compared with and , becomes closer to in the space^{2}^{2}2In spectral graph theory, smaller (or larger) eigenvalues of the associated laplacian matrix corresponds to lower (or higher) frequency domains. In Figure 1, we plot each vertex at the point , where indicates the value of the eigenvector of the laplacian matrix .. Thus, when transformed into the frequency domain, is revealed to be a more suitable choice than or for . The underlying reason is that the connectivity information of the graph has been uncovered in the frequency domain, where the relationships between vertices depend on not only their proximity but also connectivity. Thus, exploiting the spectrum of a graph can help better explore and identify the items to be recommended.Inspired by the recent progress (Kipf and Welling, 2016; Defferrard et al., 2016) in node/graph classification methods, we propose a spectral graph theory based method to leverage the broad information existing in the spectral domain to overcome the aforementioned drawbacks and challenges. Specifically, to conquer the difficulties (see Section 3.3) of directly learning from the spectral domain for recommendations, we first present a new spectral convolution operation (see Eq. (10)), which is approximated by a polynomial to dynamically amplify or attenuate each frequency domain. Then, we introduce a deep recommendation model, named Spectral Collaborative Filtering (SpectralCF), built by multiple proposed spectral convolution layers. SpectralCF directly performs collaborative filtering in the spectral domain.
The key contributions of this work can be summarized as follows:

Novelty: To the best of our knowledge, it is the first CFbased method directly learning from the spectral domains of useritem bipartite graphs.

A deep recommendation model: We propose a new spectral convolution operation performing in the spectral domain. Stacked by multiple layers of the proposed spectral convolution operation, a deep recommendation model, named Spectral Collaborative Filtering (SpectralCF), is introduced.

Strong Performance: In the experiments, SpectralCF outperforms stateoftheart comparative models. It is shown that SpectralCF effectively utilizes the rich information of connectivity existing in the spectral domain to ease the coldstart problem.
The rest of the paper is organized as follows. In Section 2, we provide preliminary concepts. Section 3 describes SpectralCF in detail. Experiments are presented in Section 4 to analyze SpectralCF and demonstrate its effectiveness compared with stateoftheart techniques for RS. In Section 5, we give a short review of the works related to our study. Finally, conclusions are presented in Section 6.
2. Definitions and Preliminaries
In this section, we present the background and preliminaries of this study. Throughout the paper, we denote scalars by either lowercase or uppercase letters, vectors by boldfaced lowercase letters, and matrices by boldfaced uppercase letters. Unless otherwise specified, all vectors are considered to be column vectors. Let
denote an identity matrix, and
and denote matrices of ones and zeros, respectively. In addition, we define the following definitions in this paper as:Definition 2.0.
(Bipartite Graph). A bipartite useritem graph with vertices and edges for recommendations is defined as , where and are two disjoint vertex sets of users and items. Every edge has the form where and and denotes that user has interacted with item in the training set.
Definition 2.0.
(Implicit Feedback Matrix). An implicit feedback matrix is a matrix defined as following:
(1) 
Definition 2.0.
(Adjacent Matrix). For the bipartite graph , its corresponding adjacent matrix can be defined as:
(2) 
where is an matrix.
Definition 2.0.
(Laplacian Matrix). The random walk laplacian matrix is defined as , where is the identity matrix and is the diagonal degree matrix defined as .
This paper focuses on the recommendation problem with implicit feedbacks, where we only observe whether a person has viewed/liked/clicked an item and do not observe explicit ratings. Let denote the set of all items liked by user and denote the remaining items. We define the recommendation problem which we study in this paper as the following:
Definition 2.0.
(Problem Definition). Given a user set and an item set , for each user who has liked/clicked/viewed an item set , we aim to recommend a ranked list of items from that are of interests to the user.
3. Proposed Model
In this section, we first describe the process of performing a
graph fourier transform
on a bipartite graph for recommendations. Then we propose to place a novel spectral convolution filter on vertices (users and items) of the bipartite graph to dynamically filter the contributions of each frequency component in the spectral domain. Later, a polynomial approximation is employed to overcome the shortcomings of the proposed convolution operation. Finally, with the approximate convolution operation, we introduce our final recommender system, named Spectral Collaborative Filtering, stacked by multiple spectral convolution layers.3.1. Graph Fourier Transform
Definition 3.0.
(Graph Signal). Given any graph , where and are a vertex and an edge set, respectively, a graph signal is defined as a state vector over all vertices in the graph, where is the value of observed at the vertex of .
The classical fourier transform is defined as an expansion of a function in terms of the complex exponentials as:
(3) 
where is an imaginary number, and the complex exponentials () form an orthonormal basis.
Analogously, the graph fourier transform is defined as an expansion of an observed graph signal in terms of the eigenvectors of the graph laplacian , and the eigenvectors serve as a basis in the spectral domain. Let us assume that a graph signal () is observed on a graph , we define the graph fourier transform and its inverse on as:
(4) 
where , and denote the , and value of , and , respectively; denotes the eigenvector of ; represents a graph signal which has been transformed into the spectral domain. For simplicity, we rewrite Eq. (4) in the matrix form as and , respectively, where are eigenvectors of .
In particular, for a bipartite graph , assume that there are two types of graph signals: and , associated with user and item vertices, respectively. We transform them into the spectral domain and vice versa as :
(5) 
3.2. Spectral Convolution Filtering
The broad information of graph structures exists in the spectral domain, and different types of connectivity information between users and items can be uncovered in different frequency domains. It is desirable to dynamically adjust the importance of each frequency domain for RS.
To this end, we propose a convolution filter, parameterized by , as into the spectral domain as:
(6) 
where and are new graph signals on learned by the filter , and denotes eigenvalues of the graph laplacian matrix .
3.3. Polynomial Approximation
Recall that we proposed a convolution operation, as shown in Eq. (6), to directly perform in the spectral domain. Although the filter is able to dynamically measure contributions of each frequency component for the purpose of recommendations, there are two limitations. First, as shown in Eq. (6), the learning complexity of the filter is , where
is the number of vertices. That is, unlike classical Convolutional Neural Networks (CNNs), the number of parameters of the filter is linear to the dimensionality of data. It constrains the scalability of the proposed filter. Second, the learned
graph signals ( and ) are vectors. It means that each vertex of users or items is represented by a scalar feature. However, a vector for every user and item is necessary to model the deep and complex connections between users and items.The first limitation can be overcome by using a polynomial approximation. We first demonstrate that the set of all convolution filters is equal to the set of finiteorder polynomials .
Proposition 3.2.
is equal to .
Proof.
Let us consider an instance . Then, . So, . Now, consider a convolution filter . Then, there must exist a polynomial function
that interpolates through all pairs
for . The maximum degree of such a polynomial is at most as there are maximum points to interpolate. Therefore, . ∎Now, we can approximate the convolution filters by using first polynomials as the following:
(7) 
In this way, the learning complexity of the filter becomes , where is a hyperparameter, and independent from the number vertices. Specially, we limit the order of the polynomial, , to 1 in order to avoid overfitting. By substituting Eq. (7) into Eq. (6), we have:
(8) 
Furthermore, it is beneficial to further decrease the number of parameters by setting . As a result, Eq. (8) becomes:
(9) 
where is a scalar.
For the second limitation, one can generalize the graph signals ( and ) to dimensional graph signals: and . Hence, Eq. (9) becomes . To take one step further, we generalize the filter parameter to a matrix of filter parameters with input channels and filters. As a result, our final spectral convolution operation is shown as the following:
(10) 
where and denote convolution results learned with filters from the spectral domain for users and items, respectively;
denotes the logistic sigmoid function.
3.4. Multilayer Model
Given user vectors and item vectors , new graph singals ( and ) in Eq. (10) are convolution results learned from the spectral domain with a parameter matrix . As in classical CNNs, one can regard Eq. (10) as a propagation rule to build a deep neural feedforward network based model, which we refer as Spectral Collaborative Filtering (SpectralCF).
Similar to word embedding techniques, we first randomly initialize user vectors and item vectors . Taking and as inputs, a layered deep spectralCF can be formulated as:
(11) 
where is a matrix of filter parameters for the layer; and denote the convolution filtering results of the layer.
In order to utilize features from all layers of SpectralCF, we further concatenate them into our final latent factors of users and items as:
(12) 
where and .
In terms of the loss function, the conventional BPR loss suggested in
(Rendle et al., 2009) is employed. BPR is a pairwise loss to address the implicit data for recommendations. Unlike pointwise based methods (Koren, 2008), BPR learns a triple , where item is liked/clicked/viewed by user and item is not. By maximizing the preference difference between and , BPR assumes that the user prefers item over the unobserved item . In particular, given a user matrix and an item matrix as shown in Eq. (12), the loss function of SpectralCF is given as:(13)  
where and denote and column of and , respectively; represents the weight on the regularization terms. The training data is generated as:
(14) 
3.5. Optimization and Prediction
At last, RMSprop
(Tieleman and Hinton, 2012) is used to minimize the loss function. The RMSprop is an adaptive version of gradient descent which adaptively controls the step size with respect to the absolute value of the gradient. It is done by scaling the updated value of each weight by a running average of its gradient norm.As shown in Algorithm 1, for a batch of randomly sampled triple
, we update parameters in each epoch using the gradients of the loss function. After the training process, with optimized
, and , we derive the user ’s preference over item as . The final item recommendation for a user is given according to the ranking criterion as Eq. (15).(15) 
4. Experiments
As discussed in the introduction section, leveraging the connectivity information in a useritem bipartite graph is essentially important for an effective recommendation model. In this section, we argue that, directly learning from the spectral domain, the proposed SpectraCF can reveal the rich information of graph structures existing in the spectral domain for making better recommendations. One may ask the following research questions:

[leftmargin=0cm]

RQ1: How much does SpectralCF benefit from the connectivity information learned from the spectral domain?

RQ2: Does SpectralCF learn from the spectral domain in an effective way?

RQ3: Compared with traditional methods, can SpectralCF better counter the coldstart problem?
In this section, in order to answer the questions above, we conduct experiments to compare SpectralCF with stateoftheart models.
4.1. Comparative Methods
To validate the effectiveness of SpectralCF, we compare it with six stateoftheart models. The comparative models can be categorized into two groups: (1) CFbased Models: To answer RQ1, we compare SpectralCF with four stateoftheart CFbased methods (ItemKNN, BPR, eALS and NCF) which ignore the information in the spectral domain; (2) Graphbased Models: For RQ2, we are interested in how effectively does SpetralCF learn the connectivity information from the spectral domain. We therefore compare SpectralCF with two graphbased models: GNMF and GCMC. Although the two models are also CFbased, we term them as graphbased models since they learn the structural information from a bipartite graph. These two groups of comparative models are summarized below:

ItemKNN (Sarwar et al., 2001): ItemKNN is a standard neighborbased collaborative filtering method. The model finds similar items for a user based on their similarities.

eALS (He et al., 2016): This is a stateoftheart matrix factorization based method for item recommendation. This model takes all unobserved interactions as negative instances and weighting them nonuniformly by the item popularity.

NCF (He et al., 2017): Neural Collaborative F
iltering fuses matrix factorization and MultiLayer Perceptron (MLP) to learn from useritem interactions. The MLP endows NCF with the ability of modelling nonlinearities between users and items.

GNMF (Cai et al., 2008): Graph regularized Nonnegative Matrix Factorization considers the graph structures by seeking a matrix factorization with a graphbased regularization.

GCMC (Berg et al., 2017): Graph Convolutional Matrix Completion utilizes a graph autoencoder to learn the connectivity information of a bipartite interaction graph for latent factors of users and items.
Please note that, GNMF and GCMC are originally designed for explicit datasets. For a fair comparison, we follow the setting of (Hu et al., 2008) to adapt them for implicit data.
4.2. Datasets
Hyperparameters  

Values  3  0.001  0.001 
We test our method as well as comparative models on three publicly available datasets^{3}^{3}3MovieLens1M and HetRec are available at https://grouplens.org/datasets/; and Amazon Instant Video can be found at http://jmcauley.ucsd.edu/data/amazon/:

MovieLens1M (Harper and Konstan, 2016): This movie rating dataset has been widely used to evaluate collaborative filtering algorithms. We used the version containing 1,000,209 ratings from 6,040 users for 3,900 movies. While it is a dataset with explicit feedbacks, we follow the convention (He et al., 2017) that transforms it into implicit data, where each entry is marked as 0 or 1 indicating whether the user has rated the item. After transforming, we retain a dataset of density.

HetRec (Cantador et al., 2011): This dataset has been released by the Second International Workshop on Information Heterogeneity and Fusion in Recommender Systems^{4}^{4}4http://ir.ii.uam.es/hetrec2011/. It is an extension of MovieLens10M dataset and contains 855,598 ratings, 2,113 users and 10,197 movies. After converting it into implicit data as MovieLens1M, we obtain a dataset of density.

Amazon Instant Video (McAuley et al., 2015): The dataset consists of 426,922 users, 23,965 videos and 583,933 ratings from Amazon.com. Similarly, we transformed it into implicit data and removed users with less than 5 interactions. As a result, a dataset of 0.12% density is obtained.
4.3. Experimental Setting
Ideally, a recommendation model should not only be able to retrieve all relevant items out of all items but also provide a rank for each user where relevant items are expected to be ranked in the top. Therefore, in our experiments, we use Recall@M and MAP@M to evaluate the performance of the topM recommendations. Recall@M is employed to measure the fraction of relevant items retrieved out of all relevant items. MAP@M is used for evaluating the ranking performance of RS. The Recall@M for each user is then defined as:
(16) 
The final results reported are average recall over all users.
For each dataset, we randomly select 80% items associated with each user to constitute the training set and use all the remaining as the test set. For each evaluation scenario, we repeat the evaluation five times with different randomly selected training sets and the average performance is reported in the following sections.
We use a validation set from the training set of each dataset to find the optimal hyperparameters of comparative methods introduced in the Section 4.1
. For ItemKNN, we employ the cosine distance to measure item similarities. The dimensions of latent factors for BPR, eALS and GNMF are searched from {8,16,32,64,128} via the validation set. The hyperparameter
of eALS is selected from 0.001 to 0.04. Since the architecture of a multilayer perceptron (MLP) is difficult to optimize, we follow the suggestion from the original paper (He et al., 2017) to employ a threelayer MLP with the shape of for NCF. The dropout rate of nodes for GCMC is searched from {0.3,0.4,0.5,0.6,0.7,0.8}. Our SpectralCF has one essential hyperparameter: . Figure 4 shows how the performances of SpectralCF vary as is set from 1 to 5 on the validation set of MovieLens1M. As we can see, in terms of Recall@20 and MAP@20, SpectralCF reaches its best performances when is fixed as 3. Other hyperparameters of SpectralCF are empirically set and summarized in Table 1, where denotes the learning rate of RMSprop. Our models are implemented in TensorFlow (Abadi et al., 2016).4.4. Experimental Results (RQ1 and RQ2)
Performance comparison in terms of recall@M with M varied from 20 to 100. Errors bars are 1standard deviation.
In Figure 5, we compare SpectralCF with four CFbased models and two graphbased models in terms of Recall@M on all three datasets. Overall, when is varied from 20 to 100, SpectralCF consistently yields the best performance across all cases. Among CFbased comparative models, ItemKNN gives the worst performances in all three datasets, indicating the necessity of modeling users’ personalized preferences rather than just recommending similar items to users. For graphbased models (GNMF and GCMC), they generally underperform CFbased models such as BPR and NCF. The unsatisfying performance of GNMF shows that adding a graphbased regularization is not sufficient to capture complex structures of graphs. Though GCMC directly performs on a useritem bipartite graph, each vertex in the graph is only allowed to learn from its neighbors. This constrains its ability of capturing global structures in the graph. Among all comparative models, benefiting from its capability of modeling nonlinear relationships between users and items, NCF beats all other models and becomes the strongest one. However, none of models above are able to directly perform in the spectral domain. They lose the rich information in the domain and as a result, SpectralCF greatly outperforms NCF by 16.1%, 16.2% and 28.0% in the dataset of MovieLen1M, HetRec and Amazon Instant Video, respectively.
In Figure 6, we compare SpectralCF with all comparative models in terms of MAP@M. Again, when is in a range from 20 to 100, SpectralCF always yields the best performance. Neighborbased ItemKNN performs the worst among all models. It further shows the advantages of modeling users’ personalized preferences. Compared with NCF and BPR, graphbased models (GNMF and GCMC) again fail to show convincing ranking performances measured by MAP@M. For CFbased models, while NCF beats other CFbased models in the dataset of HetRec, BPR shows itself as a strong model for ranking, owing to its pairwise ranking loss. It slightly outperforms NCF on average in the datasets of MovieLens1M and Amazon Instant Video. However, SpectralCF improves BPR by 15.9%, 64.9% and 47.5% in the dataset of MovieLen1M, HetRec and Amazon Instant Video, respectively.
Overall, as shown in Figure 5 and 6, not surprisingly, the performances of all models decline as the dataset becomes sparse. However, SpectralCF always outperforms all comparative models regardless of the sparsities of the datasets. By comparing spectralCF with traditional CFbased models, we demonstrate that the rich information of connectivity existing in the spectral domain assists SpectralCF in learning better latent factors of users and items. By comparing SpectralCF with graphbased models, we show that SpectralCF can effectively learn from the spectral domain.
4.5. Quality of Recommendations for Coldstart Users (RQ3)
To answer RQ3, in this section, we conduct an experiment to investigate the quality of recommendations made by SpectralCF for coldstart users. To this end, in the dataset of MovieLens1M, we build training sets with different degrees of sparsity by varying the number of items associated with each user, denoted as , from one to five. All the remaining items associated with users are used as the test set. We compare SpectralCF with BPR, which is widely known and also shown as a strong ranking performer in Figure 6. The test results are reported in the Table 2.
In Table 2, it is shown that, suffering from the coldstart problem, the performances of BPR and SpectralCF inevitably degrade. However, regardless of the number of items associated with users, SpectralCF consistently outperforms BPR in terms of Recall@20 and MAP@20. On average, SpectralCF improves BPR by 36.8% and 33.8% in Recall@20 and MAP@20, respectively. Hence, it is demonstrated that compared with BPR, spectralCF can better handle coldstart users and provide more reliable recommendations.
5. Related Works
P  1  2  3  4  5  
BPR  0.021 (0.003)  0.029 (0.004)  0.031 (0.003)  0.034 (0.004)  0.038 (0.003)  
Recall @20  SpectralCF  0.031 (0.003)  0.039 (0.003)  0.042 (0.002)  0.045 (0.003)  0.051 (0.003) 
Improve ment  47.6%  34.5%  35.5%  32.4%  34.2%  
BPR  0.014 (0.002)  0.017 (0.002)  0.021 (0.002)  0.024 (0.003)  0.027 (0.003)  
MAP @20  SpectralCF  0.019 (0.002)  0.024 (0.002)  0.028 (0.003)  0.031 (0.003)  0.035 (0.002) 
Improve ment  35.7%  41.2%  33.3%  29.2%  29.6% 
There are two categories of studies related to our work: deep learning based RS and graphbased RS. In this section, we will first briefly review existing works in the area of deep RS. Then, we focus on presenting recent works on graphbased RS. Despite all these approaches, SpectralCF is the first model to directly learn latent factors of users and items from the
spectral domains of useritem bipartite graphs.5.1. Deep Recommender Systems
One of the early works utilizing deep learning for RS builds a Restricted Boltzmann Machines (RBM) based method to model users using their rating preferences
(Salakhutdinov et al., 2007). Although the method is still a relatively shallow model, it slightly outperforms Matrix Factorization technique and shows the promising future for deep recommender systems. In (Wang et al., 2017d), a generative model and a discriminative model are employed to play a minimax game. The two models are iteratively optimized and achieve promising results for the item recommendation problem. Inspired by (Salakhutdinov et al., 2007), (Zheng et al., 2016) proposed a CF Neural Autoregressive Distribution Estimator (CFNADE) model for collaborative filtering tasks. CFNADE shares parameters between different ratings. (He et al., 2017)presents to utilize a Multilayer Perceptron (MLP) to model useritem interactions.
A number of researchers proposed to build a hybrid recommender systems to counter the sparsity problem. (Wang and Wang, 2014)
introduce Convolutional Neural Networks (CNN) and Deep Belief Network (DBN) to assist representation learning for music data. As such, their model is able to extract latent factors of songs without ratings while CF based techniques like MF are unable to handle these songs. These approaches above pretrain embeddings of users and items with matrix factorization and utilize deep models to finetune the learned item features based on item content. In
(Elkahky et al., 2015) and (Wang et al., 2017a), multiview deep models are built to utilize item information from more than one domain. (Kim et al., 2016) integrates a CNN with PMF to analyze documents associated with items to predict users’ future explicit ratings. (Zheng et al., 2017b) leverage two parallel neural networks to jointly model latent factors of users and items. To incorporate visual signals into RS, (Wang et al., 2017b)propose CNNbased models to incorporate visual signals into RS. They make use of visual features extracted from product images using deep networks to enhance the performance of RS.
(Zhang et al., 2016) investigates how to leverage the multiview information to improve the quality of recommender systems. (Cheng et al., 2016) jointly trains wide linear models and deep neural networks for video recommendations. (Wang et al., 2016) and (Zheng et al., 2017a) utilize RNN to consider word orders and extract complex semantics for recommendations. (Wang et al., 2017c) applies an attention mechanism on a sequence of models to adaptively capture the change of criteria of editors. (Zheng et al., 2018)leverages an attentional model to learn adaptive user embeddings. A survey on the deep learning based RS with more works on this topic can be found in
(Zhang et al., 2017).5.2. Graphbased Recommender Systems
In order to learn latent factors of users and items from graphs, a number of researchers have proposed graphbased RS. (Zhou et al., 2008)
develops a semisupervised learning model on graphs for document recommendation. The model combines multiple graphs in order to measure item similarities. In
(Yuan et al., 2014), they propose to model the checkin behaviors of users and a graphbased preference propagation algorithm for point of interest recommendation. The proposed solution exploits both the geographical and temporal influences in an integrated manner. (Guan et al., 2009) addresses the problem of personalized tag recommendation by modeling it as a ”query and ranking” problem. Inspired by the recent success of graph/node embedding methods, (Berg et al., 2017) proposes a graph convolution network based model for recommendations. In (Berg et al., 2017), a graph autoencoder learns the structural information of a graph for latent factors of users and items. (Cai et al., 2008) adds graphbased regularizations into the matrix factorization model to learn graph structures. Graphregularized methods are developed for the problemm of matrix completion in (Rao et al., 2015). (Monti et al., 2017)combines a convolutional neural network and a recurrent neural network to model the dynamic rating generation process. Although this work also considers the
spectral domain, they learn from a graph constructed from side information, such as genres or actors for movies. In contrast, our method learns directly from useritem bipartite graphs and does not require the side information. Thus, this work is not comparable to our method.Additionally, some scholars have proposed to incorporate the heterogeneous information on a graph for recommendations. (Jamali and Lakshmanan, 2013) suggests a general latent factor model for entities in a graph. (Yu et al., 2013) introduces a recommendation model for implicit data by taking advantage of different item similarity semantics in the graph. (Shi et al., 2015) introduces a semantic path based personalized recommendation method to predict the rating scores of users on items.
However, all works above are different from ours because they fail to consider the rich information in the spectral domains of useritem bipartite graphs. Also, our study focuses on learning from the implicit feedbacks, and leaves incorporating the heterogeneous information in a graph and the item content for future works.
6. Conclusions
It is shown that the rich information of connectivity existing in the spectral domain of a bipartite graph is helpful for discovering deep connections between users and items. In this paper, we introduce a new spectral convolution operation to directly learn latent factors of users and items from the spectral domain
. Furthermore, with the proposed operation, we build a deep feedforward neural network based recommendation model, named Spectral Collaborative Filtering (SpectralCF). Due to the rich information of connectivity existing in the
spectral domain, compared with previous works, SpectralCF is capable of discovering deep connections between users and items and therefore, alleviates the coldstart problem for CF. To the best of our knowledge, SpectralCF is the first CFbased method directly learning from the spectral domains of useritem bipartite graphs. We believe that it shows the potential of conducting CF in the spectral domain, and will encourage future works in this direction.In comparison with four stateoftheart CFbased and two graphbased models, SpectralCF achieved 20.1% and 42.6% improvements averaging on three standard datasets in terms of Recall@M and MAP@M, respectively.
Additionally, in the experiments, by varying the number of items associated with each user from 1 to 5, we build training sets with different degrees of sparsity to investigate the quality of recommendations made by SpectralCF for coldstart users. By comparing SpectralCF with BPR, on average, SpectralCF improves BPR by 36.8% and 33.8% in Recall@20 and MAP@20, respectively. It is validated that SpectralCF can effectively ameliorate the coldstart problem.
Acknowledgements.
This work is supported in part by NSF through grants IIS1526499, IIS1763325, and CNS1626432, and NSFC 61672313. This work is also partially supported by NSF through grant IIS1763365 and by FSU through the startup package and FYAP award.References
 (1)
 Abadi et al. (2016) Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. TensorFlow: A system for largescale machine learning. In Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI). Savannah, Georgia, USA.
 Berg et al. (2017) Rianne van den Berg, Thomas N Kipf, and Max Welling. 2017. Graph Convolutional Matrix Completion. arXiv preprint arXiv:1706.02263 (2017).
 Cai et al. (2008) Deng Cai, Xiaofei He, Xiaoyun Wu, and Jiawei Han. 2008. Nonnegative matrix factorization on manifold. In Data Mining, 2008. ICDM’08. Eighth IEEE International Conference on. IEEE, 63–72.
 Cantador et al. (2011) Iván Cantador, Peter Brusilovsky, and Tsvi Kuflik. 2011. 2nd Workshop on Information Heterogeneity and Fusion in Recommender Systems (HetRec 2011). In Proceedings of the 5th ACM conference on Recommender systems (RecSys 2011). ACM, New York, NY, USA.
 Cheng et al. (2016) HengTze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. 2016. Wide & deep learning for recommender systems. In Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. ACM, 7–10.
 Defferrard et al. (2016) Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems. 3844–3852.
 Elkahky et al. (2015) Ali Mamdouh Elkahky, Yang Song, and Xiaodong He. 2015. A multiview deep learning approach for cross domain user modeling in recommendation systems. In Proceedings of the 24th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 278–288.
 Gantner et al. (2011) Zeno Gantner, Steffen Rendle, Christoph Freudenthaler, and Lars SchmidtThieme. 2011. MyMediaLite: A Free Recommender System Library. In 5th ACM International Conference on Recommender Systems (RecSys 2011).
 Guan et al. (2009) Ziyu Guan, Jiajun Bu, Qiaozhu Mei, Chun Chen, and Can Wang. 2009. Personalized tag recommendation using graphbased ranking on multitype interrelated objects. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval. ACM, 540–547.
 Harper and Konstan (2016) F Maxwell Harper and Joseph A Konstan. 2016. The movielens datasets: History and context. ACM Transactions on Interactive Intelligent Systems (TiiS) 5, 4 (2016), 19.
 He et al. (2017) Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and TatSeng Chua. 2017. Neural Collaborative Filtering. In Proceedings of the 26th International Conference on World Wide Web, WWW 2017, Perth, Australia, April 37, 2017. 173–182. https://doi.org/10.1145/3038912.3052569
 He et al. (2016) Xiangnan He, Hanwang Zhang, MinYen Kan, and TatSeng Chua. 2016. Fast matrix factorization for online recommendation with implicit feedback. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. ACM, 549–558.
 Hu et al. (2008) Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Collaborative filtering for implicit feedback datasets. In 2008 Eighth IEEE International Conference on Data Mining. Ieee, 263–272.
 Jamali and Ester (2009) Mohsen Jamali and Martin Ester. 2009. Trustwalker: a random walk model for combining trustbased and itembased recommendation. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 397–406.
 Jamali and Lakshmanan (2013) Mohsen Jamali and Laks Lakshmanan. 2013. HeteroMF: recommendation in heterogeneous information networks using context dependent factor models. In Proceedings of the 22nd international conference on World Wide Web. ACM, 643–654.
 Kim et al. (2016) Donghyun Kim, Chanyoung Park, Jinoh Oh, Sungyoung Lee, and Hwanjo Yu. 2016. Convolutional matrix factorization for document contextaware recommendation. In Proceedings of the 10th ACM Conference on Recommender Systems. ACM, 233–240.
 Kipf and Welling (2016) Thomas N Kipf and Max Welling. 2016. Semisupervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).
 Koren (2008) Yehuda Koren. 2008. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 426–434.
 Koren et al. (2009) Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer 8 (2009), 30–37.
 McAuley et al. (2015) Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton van den Hengel. 2015. Imagebased Recommendations on Styles and Substitutes. arXiv preprint arXiv:1506.04757 (2015).
 Monti et al. (2017) Federico Monti, Michael Bronstein, and Xavier Bresson. 2017. Geometric matrix completion with recurrent multigraph neural networks. In Advances in Neural Information Processing Systems. 3700–3710.
 Rao et al. (2015) Nikhil Rao, HsiangFu Yu, Pradeep K Ravikumar, and Inderjit S Dhillon. 2015. Collaborative filtering with graph information: Consistency and scalable methods. In Advances in neural information processing systems. 2107–2115.

Rendle et al. (2009)
Steffen Rendle, Christoph
Freudenthaler, Zeno Gantner, and Lars
SchmidtThieme. 2009.
BPR: Bayesian personalized ranking from implicit
feedback. In
Proceedings of the twentyfifth conference on uncertainty in artificial intelligence
. AUAI Press, 452–461.  Salakhutdinov et al. (2007) Ruslan Salakhutdinov, Andriy Mnih, and Geoffrey Hinton. 2007. Restricted Boltzmann machines for collaborative filtering. In Proceedings of the 24th international conference on Machine learning. ACM, 791–798.
 Sarwar et al. (2001) Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2001. Itembased collaborative filtering recommendation algorithms. In Proceedings of the 10th international conference on World Wide Web. ACM, 285–295.
 Shi et al. (2015) Chuan Shi, Zhiqiang Zhang, Ping Luo, Philip S Yu, Yading Yue, and Bin Wu. 2015. Semantic path based personalized recommendation on weighted heterogeneous information networks. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. ACM, 453–462.

Shuman et al. (2013)
David I Shuman, Sunil K
Narang, Pascal Frossard, Antonio Ortega,
and Pierre Vandergheynst.
2013.
The emerging field of signal processing on graphs: Extending highdimensional data analysis to networks and other irregular domains.
IEEE Signal Processing Magazine 30, 3 (2013), 83–98.  Spielman (2007) Daniel A Spielman. 2007. Spectral graph theory and its applications. In Foundations of Computer Science, 2007. FOCS’07. 48th Annual IEEE Symposium on. IEEE, 29–38.
 Tieleman and Hinton (2012) Tijmen Tieleman and Geoffrey Hinton. 2012. Lecture 6.5rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning 4 (2012), 2.
 Wang et al. (2017a) Fengjiao Wang, Yongzhi Qu, Lei Zheng, ChunTa Lu, and S Yu Philip. 2017a. Deep and Broad Learning on ContentAware POI Recommendation. In Collaboration and Internet Computing (CIC), 2017 IEEE 3rd International Conference on. IEEE, 369–378.

Wang
et al. (2016)
Hao Wang, SHI Xingjian,
and DitYan Yeung. 2016.
Collaborative recurrent autoencoder: Recommend while learning to fill in the blanks. In
Advances in Neural Information Processing Systems. 415–423.  Wang et al. (2017d) Jun Wang, Lantao Yu, Weinan Zhang, Yu Gong, Yinghui Xu, Benyou Wang, Peng Zhang, and Dell Zhang. 2017d. Irgan: A minimax game for unifying generative and discriminative information retrieval models. In Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval. ACM, 515–524.
 Wang et al. (2017b) Suhang Wang, Yilin Wang, Jiliang Tang, Kai Shu, Suhas Ranganath, and Huan Liu. 2017b. What your images reveal: Exploiting visual contents for pointofinterest recommendation. In Proceedings of the 26th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 391–400.
 Wang and Wang (2014) Xinxi Wang and Ye Wang. 2014. Improving contentbased and hybrid music recommendation using deep learning. In Proceedings of the ACM International Conference on Multimedia. ACM, 627–636.
 Wang et al. (2017c) Xuejian Wang, Lantao Yu, Kan Ren, Guanyu Tao, Weinan Zhang, Yong Yu, and Jun Wang. 2017c. Dynamic attention deep model for article recommendation by learning human editors’ demonstration. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2051–2059.
 Yu et al. (2013) Xiao Yu, Xiang Ren, Yizhou Sun, Bradley Sturt, Urvashi Khandelwal, Quanquan Gu, Brandon Norick, and Jiawei Han. 2013. Recommendation in heterogeneous information networks with implicit user feedback. In Proceedings of the 7th ACM conference on Recommender systems. ACM, 347–350.
 Yuan et al. (2014) Quan Yuan, Gao Cong, and Aixin Sun. 2014. Graphbased pointofinterest recommendation with geographical and temporal influences. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management. ACM, 659–668.
 Zhang et al. (2016) Fuzheng Zhang, Nicholas Jing Yuan, Defu Lian, Xing Xie, and WeiYing Ma. 2016. Collaborative knowledge base embedding for recommender systems. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. ACM, 353–362.
 Zhang et al. (2017) Shuai Zhang, Lina Yao, and Aixin Sun. 2017. Deep learning based recommender system: A survey and new perspectives. arXiv preprint arXiv:1707.07435 (2017).
 Zheng et al. (2017a) Lei Zheng, Bokai Cao, Vahid Noroozi, S Yu Philip, and Nianzu Ma. 2017a. Hierarchical collaborative embedding for contextaware recommendations. In Big Data (Big Data), 2017 IEEE International Conference on. IEEE, 867–876.
 Zheng et al. (2018) Lei Zheng, ChunTa Lu, Lifang He, Sihong Xie, Vahid Noroozi, He Huang, and Philip S Yu. 2018. MARS: Memory AttentionAware Recommender System. arXiv preprint arXiv:1805.07037 (2018).
 Zheng et al. (2017b) Lei Zheng, Vahid Noroozi, and Philip S Yu. 2017b. Joint Deep Modeling of Users and Items Using Reviews for Recommendation. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining. ACM, 425–434.
 Zheng et al. (2016) Yin Zheng, Bangsheng Tang, Wenkui Ding, and Hanning Zhou. 2016. A neural autoregressive approach to collaborative filtering. arXiv preprint arXiv:1605.09477 (2016).
 Zhou et al. (2008) Ding Zhou, Shenghuo Zhu, Kai Yu, Xiaodan Song, Belle L Tseng, Hongyuan Zha, and C Lee Giles. 2008. Learning multiple graphs for document recommendations. In Proceedings of the 17th international conference on World Wide Web. ACM, 141–150.
Comments
There are no comments yet.