1 Introduction
With the increase of mobile devices on the market and ubiquitous presence of wireless communication networks, people gain easy access to PointofInterest (POI) recommendation services. A great number of Locationbased Social Networks (LBSNs) have consequently been established e.g., Foursquare, Gowalla, Facebook Places, and Brightkite. The LBSNs often provide POI services to recommend users new POI venues that meet specific user criteria. In this paper, we develop a high quality personalized POI recommendation system by leveraging user checkin data. There are three technical challenges listed as follows:
User checkin data sparsity. One of the major challenges is to overcome the sparsity in the user checkin data. The userPOI matrix can be extremely sparse despite of millions of POIs and users in LBSNs. Temporal reasoning. Locationbased POI recommendation systems which model personal preferences also consider the temporal context [yuan2013time, xie2016learning, griesner2015poi]. The temporal information reflects users’ needs and choices throughout the day. Spatial reasoning. A user’s current geographical location limits their choice of possible checkin POIs [liu2013personalized]. Most existing approaches model the relations between a user’s current geographical location and their preferences with respect to the surrounding POIs.

In order to address the above issues, the use of side information in traditional recommendation systems gained popularity over the past few years. Such a side information may be retrieved from social networks [lian2014geomf] and may include user demographic information, item attributes, and context information[yin2016adapting]. As the auxiliary data is useful for the recommendation systems [shi2018heterogeneous], it is desirable to model and utilize heterogeneous and complex data types in recommendation systems. However, the traditional collaborative filtering techniques such as Matrix Factorization (MF) cannot deal with the above problems in a unified manner. Knowledge graph embedding (KGE) [wang2017knowledge, cai2018comprehensive], also known as translationbased embedding model, has successfully achieved modeling the sideinformation to improve the performance of the recommender system (RS) such as [palumbo2017entity2rec, zhang2016collaborative]. He et al. [he2017translation] and Zhang et al. [zhang2016collaborative] employ KGE model to represent users, movies, and movie attributes. The graph edges represent connections between user and movies in a knowledge base [he2017translation, zhang2016collaborative]. However, previous studies do not offer insights on the following challenges: (1) how to construct a userPOI graph to utilize user checkin data with side information, such as spatialtemporal influence and semantic context information, to leverage data sparsity problem; (2) how to effectively integrate translationbased embedding technique and traditional recommendation systems to improve the preference of POI recommendation. Problem Definition: The research problem of this paper can be defined as follows: given an LBSN user checkin dataset, our target is to recommend each user with a personalized topk POIs they would be interested in visiting.
We build upon recent advances in graph embedding methods and propose Graph Embedding for POI Recommendation, a novel translationbased graph embedding approach specifically for POI recommendation, abbreviated to GERec. To overcome the challenge from the spatialtemporal context, GERec encodes temporal and spatial information, as well as user dynamic checkin activities in a lowdimensional latent space. GERec effectively addresses the issues of user checkin data sparsity by integrating userPOI graph embedding with a fused matrix factorization framework (See Fig. 2). Our contributions are:

To overcome the data sparsity problem, we propose a novel translationbased POI recommendation model, which can effectively construct a userPOI graph and model the side information, such as spatial, temporal, and semantic content information, in one graph.

We propose a spatialtemporal relation path embedding method to model the temporal, spatial and semantic content information for improving the preference of POI recommendation.

We perform experiments on two realworld LBSNs datasets collected from Foursquare [cheng2011exploring] and Gowalla [cho2011friendship]. The proposed model outperforms the stateoftheart POI recommendation approaches.
2 Related Work
POI recommendation
Making personalized POI recommendations is much more challenging due to the user dynamic checkin activities. Existing studies on MF based POI recommendations either focus on aggregating spatialaware personal preference or exploring the temporal influence on their approach. Most aggregationbased POI recommendation approaches fail to aggregate the joint effect of geographical influence, temporal influence and semantic context information as well as addressing the challenges of data sparsity problem in unified manner. In [wang2015geo, yin2017spatial], the geographical influence is used to improve POI recommendation performance. Their work highlights that there is a strong correlation between user checkin activities and geographical distance. Geographical sparse additive generative model [wang2015geo] for POI recommendation, GeoSAGE, exploited the cooccurrence pattern and the content of spatial items. In [yin2017spatial], deep representation learning for POIs from heterogeneous features and hierarchically additive representation learning for spatialaware personal preferences is proposed.
Knowledge graph embedding for recommendation
Knowledge graph embedding technique is known to enhance the performance of recommendation systems. Zhang et.al. [zhang2016collaborative] proposed a collaborative KG integrated movie recommender framework to learn the latent representation and visual representation. Palumbo et al. [palumbo2017entity2rec] implemented an application named as entity2rec to understand useritem relatedness from knowledge graph embeddings for a topN item recommendation. Qian et al. [qian2019spatiotemporal] adopts the knowledge graph embedding to model the side information in POI recommendation. Their work only focus on embedding the user and POI entities, and mapping the spatialtemporal effects as a translation matrix. Although they explored knowledge graph embeddings and RS, they did not consider the issue of incorporating the graph embedding technique with the traditional MF model. Compared with our proposed model that composites the spatial effects, temporal effects and semantic content information in a semantic relation embedding space, these KG embedding based recommender models have limited modeling and expressive abilities, since they constrain the key parameters (e.g., users’ spatial and temporal effects) to be a simple matrix.
Gaps
This paper differentiates itself from existing work as follows: 1) to the best of our knowledge, it is the first work that investigates the joint effect of temporal influence, geographical effect and semantic category information incorporating KG embedding based POI recommendation; 2) we propose a novel embedding function to bridge the gaps between the embedding technique and the traditional MF. Therefore, we propose a novel fused MF framework for dynamic userPOI preference modeling based on the learned embedding in a union manner. 3) different from the bipartite graph (homogeneous graph) based RS such as the work in [xie2016learning], our apporach focus on the translationbased graph (heterogeneous graph). Besides, [xie2016learning] does not apply MF while our model investigates the MF to predict the top POIs.
3 Proposed Approach
3.1 UserPOI graph embedding
A heterogeneous graph allows two or more node types which can be then embedded by the same function e.g., we can use a triple starting with user type or POI type without changes to the algorithm. As our goal is to focus on the recommendation performance, we want to build an effective representations for users and POIs nodes. A user and a POI represent the or of a triple , denoted as , where .
Headtail entity pairs usually exhibit diverse patterns in terms of relations [lin2015learning]
. Thus, a single relation vector cannot perform all translations from head to tail entities. For example, the relation path embedding has the diversity patterns, such as temporal and geographical patterns, and a semantic contents pattern. The relation between user (head) and POI (tail) “user  sushi shop” exhibits many patterns: (i) Temporal pattern: user visits a POI in a certain timeslot
user, /timeslot, POI; (ii) Geographical pattern: user visits a POI when she is in a particular location area user, /location, POI; and (iii) Semantic content pattern: user visits a specific POI that is associated with a category user, /category, POI. In our model, we embed spatiotemporal information as a relationship connecting users and POIs.Take the step path as an example. In Table 2, a user checkin activity, such as a user visits a POI, is usually associated with temporal and geographical information. explains that a user visits a sushi shop (POI) at pm (timeslot) at food court (location). Instead of building triples and for learning graph representation, we build a triple , and optimizes the objective . The composition operation is to join the relation with temporal effects and relation with geographical effects together into a unified relation path representation. Given a relation path , we obtain the relation path embedding r by composing multiple relations with a binary operation function, i.e., . We take the multiplication operation as the composition method. The relation path vector is calculated by accumulating the vectors of all relations for the multiplication operation, which is defined as . In our model, we embed temporal effects, geographical effects, and semantic category contents into our relation path for POI recommendation purpose. For instance, illustrates that a user visits a POI at a certain time slot in location , which has semantic category information associated with the user’s current location. We define a spatialtemporal and semanticbased relation path representation , which consists of a temporal based relation path , a geographical based relation path , and a semantic based relation path . The relation is selected to be the default relation representation in our POI recommendation model. In the rest of the paper, we use r to present for simplification.
TransR [lin2015learning, wang2017knowledge] is one of most representative transnational distance model approaches for a heterogeneous network. We apply TransR [lin2015learning] into our POI recommendation model. For each triple, including and in the network, entities are embedded into vectors and relation is embedding into . For each relation , we set a projection matrix from the entity space to the relation space. TransR firstly maps entity and into subspace of relation by using matrix :
(1) 
The TransR score function is defined as
(2) 
The following marginbased ranking loss is defined in [lin2015learning] for training
(3) 
where is the margin, and are the set of positive triples and negative triples, respectively. The existing graphs that we construct on userPOI checkin datasets only contain correct triples. We corrupt the correct triple to construct incorrect triples by replacing either head or tail entities with other entities from the same group.
(4) 
Translationbased embedding provides a general way to extract useful information from a graph. The embedding technique cannot be applied directly to matrix factorization models. Thus, we propose a function to extract the learned entities. Given an entity , and a relation , we can obtain a set of representation and , where denotes the set of relation paths. and denote the representation of user and POI with respect to the specific relation path , denote as following
(5) 
where () is the final representation for user (POI) embedding. The functiuon prepares the embedded users and POIs as the entry for matrix factorization in terms of recommendation purpose. It sorts the learned user and POI embedding set based on the calculated distance score in the descending order using Equation. (2). When a user connects with a POI by a relation (), the smaller the score value, and the lower distance between POI and user is. Hence, () vice versa. Then, the sorted user and POI embedding sets are filtered according to user current location distance. There are some specific cases that the learned POIs can be outside of her home location. It is not reasonable to recommend such POIs to a user based on her home location. To solve the geographical distance issue, a distance threshold is set up to filter the learned POIs that far away from a user home location. Following [lichman2014modeling, wang2018tpm]
, we assume a Gaussian distribution for user current location
, and set the user current checkin POI , .3.2 Integrating fused matrix factorization for recommendation
The fused matrix factorization is integrated by following two parts: 1) spatialtemporal based MF and 2) User preference based MF
. The spatialtemporal based MF calculates the probability of an embedded user will visit an embedded POI. The user preference based MF predicts user’s preference on a POI. The fused probability determine the overall probability of a user
visits a POI .3.2.1 Spatialtemporal based MF
For each embedded user and embeded POI , we apply matrix factorization to predict probability that a user would visit a POI based on her current location and a particular timeslot . Given a frequency matrix , which represents the number of checkin of the embedded users for the embedded POIs. The goal of applying matrix factorization is to find two lowrank matrices: one embedded user specific matrix and one embedded POI specific matrix , where is the dimension of the latent vector that characterizes the corresponding userPOI preference transition. The probability of a embedded user based on a particular spatialtemporal relation (), like a embedded location , is determined by
(6) 
The objective function of matrix factorization is to accurately approximate the probabilities for the user frequency data:
(7) 
where indicates the frequency of user for POI is observed; indicates the Frobenius norm of a matrix, and is a regularization term to avoid overfitting.
3.2.2 User preference based MF
The second part of the fused MF model is to predict the user preference on a POI. Based on the user historical checkin frequency, given an observed a frequency matrix , MF maps users and POIs to a shared subspace of dimensionality, such that . encodes users’ preference on a location determined by the following equation. The same objective function of matrix factorization Eq. (7) is applied to accurately approximate the probabilities for the user checkin frequency data.
(8) 
3.2.3 Fused Matrix Factorization
We propose a fused matrix factorization model that simply fusion users’ preference on a POI and the probability of whether an embedded user will visit an embedded POI together. The fused MF model, in the first part, calculates the probability of an embedded user will visit an embedded POI with spatialgeographic influence, where is defined by Eq. (6). In the second part, it predicts the user’s preference on a POI based on her historical records, where is defined in Eq. (8). The fused model is denoted as following
(9) 
4 Experiments
4.1 Experimental Configuration
Datasets We adopt two widely used largescale LBSN datasets: Foursquare [cheng2011exploring] and Gowalla [cho2011friendship]. The experiment results are compared on the same datasets for all the baseline models. We selected the Foursquare dataset from Sep 2010 to Jan 2011 which contains 1,434,668 users’ checkin activities in the USA. The Foursquare geographical space is divided into a set of location areas/regions according to administrative divisions. There are user entities and POI entities connected with spatialtemporal relations. For Gowalla, another graph is built from user entities and POI entities connected with relations. We apply means method [yin2016adapting, qian2019spatiotemporal] to cluster about regions for Gowalla geographical space.
Baselines We use the following stateoftheart methods as baselines. Two of the baseline models are translation based models that are highly related work in RS [he2017translation, qian2019spatiotemporal]. PMF [mnih2008probabilistic] is a classic probabilistic matrix factorization model by explicitly factorizing the rating matrix into two low dimensional matrices. GeoMF [lian2014geomf] is a weighted matrix factorization model for conducting POI recommendation. RankGeoFM [li2015rank] is a ranking based geographical factorization method, which considers that the checkin frequency characterizes users’ visiting preference and learn the factorization by ranking the POIs. GeoSoCa model [zhang2015geosoca]
extends the kernel density estimation by applying an adaptive bandwidth that learned from the user checkin data.
STLDA [yin2016adapting] is a latent class probabilistic generative SpatialTemporal LDA (Latent Dirichlet Allocation) model, which learns the regiondependent personal interests according to the contents of their checkedin POIs at each region. TransRec is the translationbased recommendation approach proposed in [he2017translation], which embeds items into a transition space and each models users via a translation vector. Note that our proposed method is different from TransRec as we select both users and POIs as entities, and learn the embedding representation for a different type of knowledge as well as the spatialtemporal relationships. STA [qian2019spatiotemporal] is a spatialtemporal contextaware and translationbased POI recommendation model. However, this solution does not consider the semantic relation embedding of spatial, temporal and category content information, and thus is incapable of leveraging the userPOI graph structure information.Evaluation Metrics Following [lian2014geomf, li2015rank, zhang2015geosoca], we deploy the following evaluation methodology. The userPOI graph is built on historical user checkin activities from the training set. The spatialtemporal relation in the userPOI graph is composed based on each user’s current time slot and location area from the given query . We divide the time slot to different hour length . The user’s current standing position area before visiting is select for presenting location . We use the as the cutoff point so that checkins before a particular date are used for training. The rest checkin data generated after this date is chosen for testing. We form a top recommendation list from the top POI recommendations. We deploy the following measurement metrics, including Precision (), Recall () and F1score (): and .
4.2 Effectiveness experiments
Following [lin2015learning, lin2015modeling], for translational distance model TransR, we set the learning rate , the margin , the dimensions of entity embedding and relation embedding , the batch size . We traverse all the training triples for 1000 rounds for both Foursquare and Gowalla datasets. Fig. 3 report the performance of the POI recommendation models on Foursquare and Gowalla datasets, respectively. We present the performance when . Fig. 3 presents the results of compared algorithms in terms of , and on Foursquare and Gowalla datasets. It is observed that the proposed GERec model outperforms all baseline models significantly in all measurement metrics at different values on both datasets. Specifically, when comparing with the traditional MF models, GERec outperforms the RankGeoMF, which is the MF baseline with the best performance, by 50% and 47% in F1score@10 on Foursquare and Gowalla, respectively. When comparing with translationbased models, our proposed model also improves the POI recommendation performance significantly. GERec outperforms STA, by 20% and 25% in F1score@10 on both datasets. These demonstrates the capability of our graphbased model GERec to generate high quality POI recommendations. Although GeoSoCa exploits social influence, geographical effect and user interest, the simple kernel density estimation results in the poor performance. This validates the effectiveness of our GERec solution, especially our proposed framework to exploit and integrate the userPOI interaction and spatialtemporal information to overcome the sparsity issue of the userPOI checkin data in learning user embedding, spatialtemporal and semantic content influence, respectively. The learned embeddings are well integrated into the fused matrix factorization model by our proposed Embedding Fused Function. Thus, GERec achieves the best performance among all compared baseline models. The userPOI graph constructed from Gowalla dataset has fewer relation edges than Foursquare graph, as Gowalla relation pattern has less region in the relation path than Foursquare.
4.3 Effect of data sparsity
In Fig. 4, we conduct extensive experiments to evaluate the performance of the models for the data sparsity problem. Specifically, we create multiple datasets with various sparsity levels by reducing the amount of training data randomly by 10%, 20%, 30%, and 40% of the total amount of data (before the cutoff date), and at the same time keeping the test data the same. The results on both Foursquare and Gowalla are shown in the Fig. 4. Specifically, the in the horizontal axis presents the experiment result without reducing training data. We observe that the Precision@ and Recall@ value decrease for all baseline models. For example, the performances of these models in terms of Prec@10 decrease at least 40% on Foursquare, when reducing 40% of the training data. The STLDA results drop significantly compared with the other baseline models with 37% drop in Rec@10 on both Foursquare and Gowalla, which indicates that the LDAbased model is sensitive when the data becomes sparse. The PMF does not change much, however, it remains the worst accurate result. The Rec@10 values of RankGeoMF, TransRec, and STA show a 42%, 41%, and 37% drop, respectively. The GERec changes 34% only, which illustrates that our proposed model is more stable and robust for the data sparsity problem than the baselines.
4.4 Impact of time slot and dimensionality
There are two parameters in the proposed GERec model: the time slot and the embedding dimension . Here, we investigate the effect of these two parameters. Table 1 shows the impact of the length of time slot in Precision@ and Recall@. The length of the time slot affects the performance of POI recommendation. When the length of time slot changes, the relation path is recomposed from user’s checkin time slot and current location. Thus, the entire structure of the graph is reconstructed. We split a day activities into different length. The parameter demonstrates how many hours in each time slot. The larger length of time slot indicates less time influence on recommendation results. We report the top
recommendation precision and recall for each time slot on the Foursquare and Gowalla datasets. From the experiment results, we observe that the POI recommendation accuracy is improved when the time slot length increases. The recommendation accuracy reaches a peak point at around
hours time slot. Then, it starts decreasing when the timeslot keeps increasing. The reason for better accuracy is that the larger time length, the denser data will be. Hence, there are more user checkin records at each time slot for recommendation purpose. However, the recommendation accuracy decreases when the length of the time slot reach hours. This is because when the length of the time slot is large enough, it may reduce the influence of temporal effects. Moreover, we study the impact of varying dimension parameters. The dimensions of relation embedding is set among (as shown in Table 2). The best parameter is determined according to the mean rank in the test set. The accuracy rate increases gradually when the dimension increases. Specifically, the accuracy keeps increasing until the dimension reaches , then tends to be stable.Hours  Foursquare  Gowalla  
Prec@1  Prec@10  Prec@20  Rec@1  Rec@10  Rec@20  Prec@1  Prec@10  Prec@20  Rec@1  Rec@10  Rec@20  
1  0.075  0.061  0.041  0.062  0.128  0.163  0.089  0.064  0.047  0.071  0.141  0.169 
2  0.100  0.083  0.054  0.082  0.170  0.218  0.119  0.086  0.062  0.094  0.150  0.225 
4  0.113  0.090  0.060  0.092  0.192  0.245  0.134  0.096  0.070  0.106  0.168  0.253 
8  0.125  0.100  0.067  0.103  0.213  0.272  0.149  0.107  0.078  0.118  0.187  0.281 
12  0.119  0.095  0.064  0.098  0.202  0.258  0.142  0.102  0.074  0.112  0.178  0.267 
24  0.115  0.092  0.062  0.094  0.196  0.250  0.137  0.098  0.072  0.108  0.172  0.259 
Foursquare  Gowalla  
Prec@1  Prec@10  Prec@20  Rec@1  Rec@10  Rec@20  Prec@1  Prec@10  Prec@20  Rec@1  Rec@10  Rec@20  
70  0.121  0.097  0.065  0.099  0.206  0.262  0.143  0.123  0.075  0.113  0.226  0.270 
80  0.123  0.098  0.066  0.101  0.209  0.267  0.146  0.125  0.076  0.115  0.183  0.275 
90  0.124  0.099  0.066  0.102  0.211  0.269  0.148  0.127  0.077  0.117  0.185  0.278 
100  0.125  0.100  0.067  0.103  0.213  0.272  0.149  0.128  0.078  0.118  0.187  0.281 
110  0.126  0.100  0.067  0.103  0.214  0.273  0.150  0.129  0.078  0.118  0.188  0.282 
120  0.126  0.101  0.067  0.103  0.214  0.274  0.150  0.129  0.079  0.119  0.188  0.283 
5 Conclusion
In this paper, we propose a novel translationbased POI recommendation model, which can effectively construct a userPOI graph and model the side information. To address time reasoning and geographical reasoning problems, we propose a spatialtemporal relation path embedding method to model the temporal, spatial and semantic content information to leverage the userPOI interaction data to improve the learning of user embedding. To overcome the sparsity of the userPOI interaction data, we develop an embedding function to bridge the gaps between the translationbased embedding model and traditional MFbased model. The userPOI graph is integrated with a fused MF model for improving the preference of POI recommendation.
Acknowledgment
This research is supported by Australian Research Council Discovery Project DP190101485, Alexander von Humboldt Foundation, Bayer Foundation, and CSIRO Data61 Scholarship program.
Comments
There are no comments yet.