With the increase of mobile devices on the market and ubiquitous presence of wireless communication networks, people gain easy access to Point-of-Interest (POI) recommendation services. A great number of Location-based Social Networks (LBSNs) have consequently been established e.g., Foursquare, Gowalla, Facebook Places, and Brightkite. The LBSNs often provide POI services to recommend users new POI venues that meet specific user criteria. In this paper, we develop a high quality personalized POI recommendation system by leveraging user check-in data. There are three technical challenges listed as follows:
User check-in data sparsity. One of the major challenges is to overcome the sparsity in the user check-in data. The user-POI matrix can be extremely sparse despite of millions of POIs and users in LBSNs. Temporal reasoning. Location-based POI recommendation systems which model personal preferences also consider the temporal context [yuan2013time, xie2016learning, griesner2015poi]. The temporal information reflects users’ needs and choices throughout the day. Spatial reasoning. A user’s current geographical location limits their choice of possible check-in POIs [liu2013personalized]. Most existing approaches model the relations between a user’s current geographical location and their preferences with respect to the surrounding POIs.
In order to address the above issues, the use of side information in traditional recommendation systems gained popularity over the past few years. Such a side information may be retrieved from social networks [lian2014geomf] and may include user demographic information, item attributes, and context information[yin2016adapting]. As the auxiliary data is useful for the recommendation systems [shi2018heterogeneous], it is desirable to model and utilize heterogeneous and complex data types in recommendation systems. However, the traditional collaborative filtering techniques such as Matrix Factorization (MF) cannot deal with the above problems in a unified manner. Knowledge graph embedding (KGE) [wang2017knowledge, cai2018comprehensive], also known as translation-based embedding model, has successfully achieved modeling the side-information to improve the performance of the recommender system (RS) such as [palumbo2017entity2rec, zhang2016collaborative]. He et al. [he2017translation] and Zhang et al. [zhang2016collaborative] employ KGE model to represent users, movies, and movie attributes. The graph edges represent connections between user and movies in a knowledge base [he2017translation, zhang2016collaborative]. However, previous studies do not offer insights on the following challenges: (1) how to construct a user-POI graph to utilize user check-in data with side information, such as spatial-temporal influence and semantic context information, to leverage data sparsity problem; (2) how to effectively integrate translation-based embedding technique and traditional recommendation systems to improve the preference of POI recommendation. Problem Definition: The research problem of this paper can be defined as follows: given an LBSN user check-in dataset, our target is to recommend each user with a personalized top-k POIs they would be interested in visiting.
We build upon recent advances in graph embedding methods and propose Graph Embedding for POI Recommendation, a novel translation-based graph embedding approach specifically for POI recommendation, abbreviated to GERec. To overcome the challenge from the spatial-temporal context, GERec encodes temporal and spatial information, as well as user dynamic check-in activities in a low-dimensional latent space. GERec effectively addresses the issues of user check-in data sparsity by integrating user-POI graph embedding with a fused matrix factorization framework (See Fig. 2). Our contributions are:
To overcome the data sparsity problem, we propose a novel translation-based POI recommendation model, which can effectively construct a user-POI graph and model the side information, such as spatial, temporal, and semantic content information, in one graph.
We propose a spatial-temporal relation path embedding method to model the temporal, spatial and semantic content information for improving the preference of POI recommendation.
We perform experiments on two real-world LBSNs datasets collected from Foursquare [cheng2011exploring] and Gowalla [cho2011friendship]. The proposed model outperforms the state-of-the-art POI recommendation approaches.
2 Related Work
Making personalized POI recommendations is much more challenging due to the user dynamic check-in activities. Existing studies on MF based POI recommendations either focus on aggregating spatial-aware personal preference or exploring the temporal influence on their approach. Most aggregation-based POI recommendation approaches fail to aggregate the joint effect of geographical influence, temporal influence and semantic context information as well as addressing the challenges of data sparsity problem in unified manner. In [wang2015geo, yin2017spatial], the geographical influence is used to improve POI recommendation performance. Their work highlights that there is a strong correlation between user check-in activities and geographical distance. Geographical sparse additive generative model [wang2015geo] for POI recommendation, Geo-SAGE, exploited the co-occurrence pattern and the content of spatial items. In [yin2017spatial], deep representation learning for POIs from heterogeneous features and hierarchically additive representation learning for spatial-aware personal preferences is proposed.
Knowledge graph embedding for recommendation
Knowledge graph embedding technique is known to enhance the performance of recommendation systems. Zhang et.al. [zhang2016collaborative] proposed a collaborative KG integrated movie recommender framework to learn the latent representation and visual representation. Palumbo et al. [palumbo2017entity2rec] implemented an application named as entity2rec to understand user-item relatedness from knowledge graph embeddings for a top-N item recommendation. Qian et al. [qian2019spatiotemporal] adopts the knowledge graph embedding to model the side information in POI recommendation. Their work only focus on embedding the user and POI entities, and mapping the spatial-temporal effects as a translation matrix. Although they explored knowledge graph embeddings and RS, they did not consider the issue of incorporating the graph embedding technique with the traditional MF model. Compared with our proposed model that composites the spatial effects, temporal effects and semantic content information in a semantic relation embedding space, these KG embedding based recommender models have limited modeling and expressive abilities, since they constrain the key parameters (e.g., users’ spatial and temporal effects) to be a simple matrix.
This paper differentiates itself from existing work as follows: 1) to the best of our knowledge, it is the first work that investigates the joint effect of temporal influence, geographical effect and semantic category information incorporating KG embedding based POI recommendation; 2) we propose a novel embedding function to bridge the gaps between the embedding technique and the traditional MF. Therefore, we propose a novel fused MF framework for dynamic user-POI preference modeling based on the learned embedding in a union manner. 3) different from the bipartite graph (homogeneous graph) based RS such as the work in [xie2016learning], our apporach focus on the translation-based graph (heterogeneous graph). Besides, [xie2016learning] does not apply MF while our model investigates the MF to predict the top- POIs.
3 Proposed Approach
3.1 User-POI graph embedding
A heterogeneous graph allows two or more node types which can be then embedded by the same function e.g., we can use a triple starting with user type or POI type without changes to the algorithm. As our goal is to focus on the recommendation performance, we want to build an effective representations for users and POIs nodes. A user and a POI represent the or of a triple , denoted as , where .
Head-tail entity pairs usually exhibit diverse patterns in terms of relations [lin2015learning]
. Thus, a single relation vector cannot perform all translations from head to tail entities. For example, the relation path embedding has the diversity patterns, such as temporal and geographical patterns, and a semantic contents pattern. The relation between user (head) and POI (tail) “user - sushi shop” exhibits many patterns: (i) Temporal pattern: user visits a POI in a certain timeslotuser, /timeslot, POI; (ii) Geographical pattern: user visits a POI when she is in a particular location area user, /location, POI; and (iii) Semantic content pattern: user visits a specific POI that is associated with a category user, /category, POI. In our model, we embed spatio-temporal information as a relationship connecting users and POIs.
Take the -step path as an example. In Table 2, a user check-in activity, such as a user visits a POI, is usually associated with temporal and geographical information. explains that a user visits a sushi shop (POI) at pm (timeslot) at food court (location). Instead of building triples and for learning graph representation, we build a triple , and optimizes the objective . The composition operation is to join the relation with temporal effects and relation with geographical effects together into a unified relation path representation. Given a relation path , we obtain the relation path embedding r by composing multiple relations with a binary operation function, i.e., . We take the multiplication operation as the composition method. The relation path vector is calculated by accumulating the vectors of all relations for the multiplication operation, which is defined as . In our model, we embed temporal effects, geographical effects, and semantic category contents into our relation path for POI recommendation purpose. For instance, illustrates that a user visits a POI at a certain time slot in location , which has semantic category information associated with the user’s current location. We define a spatial-temporal and semantic-based relation path representation , which consists of a temporal based relation path , a geographical based relation path , and a semantic based relation path . The relation is selected to be the default relation representation in our POI recommendation model. In the rest of the paper, we use r to present for simplification.
TransR [lin2015learning, wang2017knowledge] is one of most representative transnational distance model approaches for a heterogeneous network. We apply TransR [lin2015learning] into our POI recommendation model. For each triple, including and in the network, entities are embedded into vectors and relation is embedding into . For each relation , we set a projection matrix from the entity space to the relation space. TransR firstly maps entity and into subspace of relation by using matrix :
The TransR score function is defined as
The following margin-based ranking loss is defined in [lin2015learning] for training
where is the margin, and are the set of positive triples and negative triples, respectively. The existing graphs that we construct on user-POI check-in datasets only contain correct triples. We corrupt the correct triple to construct incorrect triples by replacing either head or tail entities with other entities from the same group.
Translation-based embedding provides a general way to extract useful information from a graph. The embedding technique cannot be applied directly to matrix factorization models. Thus, we propose a function to extract the learned entities. Given an entity , and a relation , we can obtain a set of representation and , where denotes the set of relation paths. and denote the representation of user and POI with respect to the specific relation path , denote as following
where () is the final representation for user (POI) embedding. The functiuon prepares the embedded users and POIs as the entry for matrix factorization in terms of recommendation purpose. It sorts the learned user and POI embedding set based on the calculated distance score in the descending order using Equation. (2). When a user connects with a POI by a relation (), the smaller the score value, and the lower distance between POI and user is. Hence, () vice versa. Then, the sorted user and POI embedding sets are filtered according to user current location distance. There are some specific cases that the learned POIs can be outside of her home location. It is not reasonable to recommend such POIs to a user based on her home location. To solve the geographical distance issue, a distance threshold is set up to filter the learned POIs that far away from a user home location. Following [lichman2014modeling, wang2018tpm]
, we assume a Gaussian distribution for user current location, and set the user current check-in POI , .
3.2 Integrating fused matrix factorization for recommendation
The fused matrix factorization is integrated by following two parts: 1) spatial-temporal based MF and 2) User preference based MF
. The spatial-temporal based MF calculates the probability of an embedded user will visit an embedded POI. The user preference based MF predicts user’s preference on a POI. The fused probability determine the overall probability of a uservisits a POI .
3.2.1 Spatial-temporal based MF
For each embedded user and embeded POI , we apply matrix factorization to predict probability that a user would visit a POI based on her current location and a particular timeslot . Given a frequency matrix , which represents the number of check-in of the embedded users for the embedded POIs. The goal of applying matrix factorization is to find two low-rank matrices: one embedded user specific matrix and one embedded POI specific matrix , where is the dimension of the latent vector that characterizes the corresponding user-POI preference transition. The probability of a embedded user based on a particular spatial-temporal relation (), like a embedded location , is determined by
The objective function of matrix factorization is to accurately approximate the probabilities for the user frequency data:
where indicates the frequency of user for POI is observed; indicates the Frobenius norm of a matrix, and is a regularization term to avoid overfitting.
3.2.2 User preference based MF
The second part of the fused MF model is to predict the user preference on a POI. Based on the user historical check-in frequency, given an observed a frequency matrix , MF maps users and POIs to a shared subspace of dimensionality, such that . encodes users’ preference on a location determined by the following equation. The same objective function of matrix factorization Eq. (7) is applied to accurately approximate the probabilities for the user check-in frequency data.
3.2.3 Fused Matrix Factorization
We propose a fused matrix factorization model that simply fusion users’ preference on a POI and the probability of whether an embedded user will visit an embedded POI together. The fused MF model, in the first part, calculates the probability of an embedded user will visit an embedded POI with spatial-geographic influence, where is defined by Eq. (6). In the second part, it predicts the user’s preference on a POI based on her historical records, where is defined in Eq. (8). The fused model is denoted as following
4.1 Experimental Configuration
Datasets We adopt two widely used large-scale LBSN datasets: Foursquare [cheng2011exploring] and Gowalla [cho2011friendship]. The experiment results are compared on the same datasets for all the baseline models. We selected the Foursquare dataset from Sep 2010 to Jan 2011 which contains 1,434,668 users’ check-in activities in the USA. The Foursquare geographical space is divided into a set of location areas/regions according to administrative divisions. There are user entities and POI entities connected with spatial-temporal relations. For Gowalla, another graph is built from user entities and POI entities connected with relations. We apply -means method [yin2016adapting, qian2019spatiotemporal] to cluster about regions for Gowalla geographical space.
Baselines We use the following state-of-the-art methods as baselines. Two of the baseline models are translation based models that are highly related work in RS [he2017translation, qian2019spatiotemporal]. PMF [mnih2008probabilistic] is a classic probabilistic matrix factorization model by explicitly factorizing the rating matrix into two low- dimensional matrices. GeoMF [lian2014geomf] is a weighted matrix factorization model for conducting POI recommendation. Rank-GeoFM [li2015rank] is a ranking based geographical factorization method, which considers that the check-in frequency characterizes users’ visiting preference and learn the factorization by ranking the POIs. GeoSoCa model [zhang2015geosoca]
extends the kernel density estimation by applying an adaptive bandwidth that learned from the user check-in data.ST-LDA [yin2016adapting] is a latent class probabilistic generative Spatial-Temporal LDA (Latent Dirichlet Allocation) model, which learns the region-dependent personal interests according to the contents of their checked-in POIs at each region. TransRec is the translation-based recommendation approach proposed in [he2017translation], which embeds items into a transition space and each models users via a translation vector. Note that our proposed method is different from TransRec as we select both users and POIs as entities, and learn the embedding representation for a different type of knowledge as well as the spatial-temporal relationships. STA [qian2019spatiotemporal] is a spatial-temporal context-aware and translation-based POI recommendation model. However, this solution does not consider the semantic relation embedding of spatial, temporal and category content information, and thus is incapable of leveraging the user-POI graph structure information.
Evaluation Metrics Following [lian2014geomf, li2015rank, zhang2015geosoca], we deploy the following evaluation methodology. The user-POI graph is built on historical user check-in activities from the training set. The spatial-temporal relation in the user-POI graph is composed based on each user’s current time slot and location area from the given query . We divide the time slot to different hour length . The user’s current standing position area before visiting is select for presenting location . We use the as the cut-off point so that check-ins before a particular date are used for training. The rest check-in data generated after this date is chosen for testing. We form a top- recommendation list from the top POI recommendations. We deploy the following measurement metrics, including Precision (), Recall () and F1-score (): and .
4.2 Effectiveness experiments
Following [lin2015learning, lin2015modeling], for translational distance model TransR, we set the learning rate , the margin , the dimensions of entity embedding and relation embedding , the batch size . We traverse all the training triples for 1000 rounds for both Foursquare and Gowalla datasets. Fig. 3 report the performance of the POI recommendation models on Foursquare and Gowalla datasets, respectively. We present the performance when . Fig. 3 presents the results of compared algorithms in terms of , and on Foursquare and Gowalla datasets. It is observed that the proposed GERec model outperforms all baseline models significantly in all measurement metrics at different values on both datasets. Specifically, when comparing with the traditional MF models, GERec outperforms the Rank-GeoMF, which is the MF baseline with the best performance, by 50% and 47% in F1-score@10 on Foursquare and Gowalla, respectively. When comparing with translation-based models, our proposed model also improves the POI recommendation performance significantly. GERec outperforms STA, by 20% and 25% in F1-score@10 on both datasets. These demonstrates the capability of our graph-based model GERec to generate high quality POI recommendations. Although GeoSoCa exploits social influence, geographical effect and user interest, the simple kernel density estimation results in the poor performance. This validates the effectiveness of our GERec solution, especially our proposed framework to exploit and integrate the user-POI interaction and spatial-temporal information to overcome the sparsity issue of the user-POI check-in data in learning user embedding, spatial-temporal and semantic content influence, respectively. The learned embeddings are well integrated into the fused matrix factorization model by our proposed Embedding Fused Function. Thus, GERec achieves the best performance among all compared baseline models. The user-POI graph constructed from Gowalla dataset has fewer relation edges than Foursquare graph, as Gowalla relation pattern has less region in the relation path than Foursquare.
4.3 Effect of data sparsity
In Fig. 4, we conduct extensive experiments to evaluate the performance of the models for the data sparsity problem. Specifically, we create multiple datasets with various sparsity levels by reducing the amount of training data randomly by 10%, 20%, 30%, and 40% of the total amount of data (before the cut-off date), and at the same time keeping the test data the same. The results on both Foursquare and Gowalla are shown in the Fig. 4. Specifically, the in the horizontal axis presents the experiment result without reducing training data. We observe that the Precision@ and Recall@ value decrease for all baseline models. For example, the performances of these models in terms of Prec@10 decrease at least 40% on Foursquare, when reducing 40% of the training data. The ST-LDA results drop significantly compared with the other baseline models with 37% drop in Rec@10 on both Foursquare and Gowalla, which indicates that the LDA-based model is sensitive when the data becomes sparse. The PMF does not change much, however, it remains the worst accurate result. The Rec@10 values of RankGeoMF, TransRec, and STA show a 42%, 41%, and 37% drop, respectively. The GERec changes 34% only, which illustrates that our proposed model is more stable and robust for the data sparsity problem than the baselines.
4.4 Impact of time slot and dimensionality
There are two parameters in the proposed GERec model: the time slot and the embedding dimension . Here, we investigate the effect of these two parameters. Table 1 shows the impact of the length of time slot in Precision@ and Recall@. The length of the time slot affects the performance of POI recommendation. When the length of time slot changes, the relation path is re-composed from user’s check-in time slot and current location. Thus, the entire structure of the graph is reconstructed. We split a day activities into different length. The parameter demonstrates how many hours in each time slot. The larger length of time slot indicates less time influence on recommendation results. We report the top-
recommendation precision and recall for each time slot on the Foursquare and Gowalla datasets. From the experiment results, we observe that the POI recommendation accuracy is improved when the time slot length increases. The recommendation accuracy reaches a peak point at aroundhours time slot. Then, it starts decreasing when the time-slot keeps increasing. The reason for better accuracy is that the larger time length, the denser data will be. Hence, there are more user check-in records at each time slot for recommendation purpose. However, the recommendation accuracy decreases when the length of the time slot reach hours. This is because when the length of the time slot is large enough, it may reduce the influence of temporal effects. Moreover, we study the impact of varying dimension parameters. The dimensions of relation embedding is set among (as shown in Table 2). The best parameter is determined according to the mean rank in the test set. The accuracy rate increases gradually when the dimension increases. Specifically, the accuracy keeps increasing until the dimension reaches , then tends to be stable.
In this paper, we propose a novel translation-based POI recommendation model, which can effectively construct a user-POI graph and model the side information. To address time reasoning and geographical reasoning problems, we propose a spatial-temporal relation path embedding method to model the temporal, spatial and semantic content information to leverage the user-POI interaction data to improve the learning of user embedding. To overcome the sparsity of the user-POI interaction data, we develop an embedding function to bridge the gaps between the translation-based embedding model and traditional MF-based model. The user-POI graph is integrated with a fused MF model for improving the preference of POI recommendation.
This research is supported by Australian Research Council Discovery Project DP190101485, Alexander von Humboldt Foundation, Bayer Foundation, and CSIRO Data61 Scholarship program.