TrajectoryNet: An Embedded GPS Trajectory Representation for Point-based Classification Using Recurrent Neural Networks

by   Xiang Jiang, et al.
Acadia University
Dalhousie University

Understanding and discovering knowledge from GPS (Global Positioning System) traces of human activities is an essential topic in mobility-based urban computing. We propose TrajectoryNet-a neural network architecture for point-based trajectory classification to infer real world human transportation modes from GPS traces. To overcome the challenge of capturing the underlying latent factors in the low-dimensional and heterogeneous feature space imposed by GPS data, we develop a novel representation that embeds the original feature space into another space that can be understood as a form of basis expansion. We also enrich the feature space via segment-based information and use Maxout activations to improve the predictive power of Recurrent Neural Networks (RNNs). We achieve over 98 of transportation modes, outperforming existing models without additional sensory data or location-based prior knowledge.



page 1

page 2

page 3

page 4


Predicting Transportation Modes of GPS Trajectories using Feature Engineering and Noise Removal

Understanding transportation mode from GPS (Global Positioning System) t...

Transportation Modes Classification Using Feature Engineering

Predicting transportation modes from GPS (Global Positioning System) rec...

Inferring Taxi Status Using GPS Trajectories

In this paper, we infer the statuses of a taxi, consisting of occupied, ...

Inferring transportation modes from GPS trajectories using a convolutional neural network

Identifying the distribution of users' transportation modes is an essent...

Leveraging the Self-Transition Probability of Ordinal Pattern Transition Graph for Transportation Mode Classification

The analysis of GPS trajectories is a well-studied problem in Urban Comp...

A Simple Sinuosity-Based Method using GPS data to Support Mitigation Policies for Public Buses GHG Emissions

It is clear by now that climate change mitigation relies on our capacity...

Towards a Drone Cinematographer: Guiding Quadrotor Cameras using Visual Composition Principles

We present a system to capture video footage of human subjects in the re...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1. Introduction

The advent of ubiquitous location-acquisition technologies, such as GPS and AIS (Automatic Identification System), has enabled massive collection of spatiotemporal trajectory data. Understanding and discovering knowledge from GPS and AIS data allows us to draw a global picture of human activities and improve our relationship with the planet earth. Among their many applications, trajectory data mining algorithms search for patterns to cluster, forecast or classify a variety of moving objects, including animals, human, cars, and vessels

(Giannotti et al., 2007; Zheng, 2015; Zheng et al., 2014; Batty et al., 2012; Kisilevich et al., 2010; Nanni and Pedreschi, 2006; Loglisci et al., 2014; Dodge et al., 2009). Such applications include time series forecasting tasks such as predicting the flow of crowds (Zhang et al., 2016b, a) and time series classification tasks such as detecting human transportation modes (Zheng et al., 2008a) and fishing activities (de Souza et al., 2016; Hu et al., 2016; Jiang et al., 2017). These applications allow us to improve traffic management, public safety, and environmental sustainability. In this paper, we investigate the transportation mode detection task using Recurrent Neural Networks (RNNs) that classify GPS traces into four classes (i.e., bike, car, walk and bus).

The reason why neural networks (Byon et al., 2009; Gonzalez et al., 2010) fail to achieve highly accurate models on this task is due to the difficulty of developing hierarchies of feature compositions in low-dimensional and heterogeneous feature space. To address these issues, we extend our previous work (Jiang et al., 2017) on RNNs beyond ocean data to detect human transportation modes from GPS traces. There is a significant novelty over (Jiang et al., 2017): an in-depth analysis of the embedding method and a thorough investigation of its connections to basis expansion, piecewise function and discretization. Also, for the first time, we put forth the positive use of Maxout in GRUs as universal approximators.

We propose the TrajectoryNet method that achieves state-of-the-art performance on real world GPS transportation mode classification tasks. The proposed TrajectoryNet differs with existing methods in that it uses embedding of GPS data, for the first time, to map the low-dimensional and heterogeneous feature space into distributed vector representations to capture the high-level semantics

111Note that the term “semantic” in this paper refers to meaningful representations of the data, rather than geo-objects such as roads and places of interest.. The embedding can be viewed as a form of basis expansion that improves feature representation in the way that even a linear decision boundary in the embedding space can be mapped down to a highly nonlinear function in the original feature space. We also employ segment-based information and Maxout activations (Goodfellow et al., 2013) to improve the predictive power of RNNs. The TrajectoryNet achieves over 98% and 97% classification accuracy when detecting 4 and 7 types of transportation modes.

The rest of this paper is structured as follows: In Section 2

we provide definitions about trajectory data mining. We also introduce RNNs especially Gated Recurrent Units (GRU) that will be used in this paper. In Section

3, we introduce the framework of the proposed model—TrajectoryNet. We detail the segmentation method that defines the neighbourhood, embedding method and the Maxout GRU classification model. We highlight the relationship between embedding and discretization in neural networks and provide intuitive justifications about the need of embedding for continuous features. In Section 4, we detail experiment settings and discussions on the experimental results. We summarize the conclusion and future work in Section 5.

2. Preliminaries

2.1. Definitions

Definition 2.1 ().

A trajectory (Devogele et al., 2012) is a part of movement of an object that is delimited by a given time interval . It is a continuous function from time to space.

Definition 2.2 ().

A discrete representation (Devogele et al., 2012; Giannotti et al., 2007) of a trajectory is made up of a sequence of triples that represents spatio-temporal positions of the trajectory, but not providing the continuity of the movement of the object. Here denotes the spatial coordinate at time .

The discrete representation is due to the sampling nature of location-acquisition technologies where the trajectory data are sampled at discrete timestamps.

Definition 2.3 ().

Point-based classification of a trajectory is the practice of learning a one-to-one mapping that maps a sequence of discrete trajectory data to a corresponding sequence of labels where denotes the class label of triple .

Definition 2.4 ().

Segmentation (Giannotti et al., 2007) of a trajectory is to divide a trajectory into disjoint segments with some criteria such as time interval, trajectory shape or semantics that can provide richer knowledge from trajectory data.

Definition 2.5 ().

Discretization of continuous features (Dougherty et al., 1995; Kotsiantis and Kanellopoulos, 2006; Garcia et al., 2013; Ali et al., 2015) divides the domain of the continuous attribute into a set of intervals using cut-points represented by where . The domain is divided into disjoint intervals where , are the lower and upper bounds of this attribute.

2.2. Recurrent Neural Networks

RNN is a powerful model for learning from sequential data. GPS trajectories are a type of spatiotemporal data that naturally fits into the framework of RNNs. Unlike standard feedforward neural networks, RNNs use recurrent connections to retain the state information between different time steps. Long short-term memory networks (LSTMs) are introduced to overcome the optimization challenges in RNNs

(Bengio et al., 1994; Hochreiter, 1991, 1998) with the use of a sophisticated network structure that selectively passes information at different time steps. There is a rich family of LSTM architectures (Greff et al., 2015; Chung et al., 2014), and our recent work (Jiang et al., 2017) suggests that Gated Recurrent Units (GRU) (Cho et al., 2014) are better suited for point-based trajectory classification. Gated Recurrent Units (Cho et al., 2014) defined in Equation (1) are a variant of LSTM.


where , are the reset and update gates to learn short and long-term memories, and are candidate and final cell states at , and are input-to-hidden and recurrent connections and

denotes element-wise multiplication. Compared with LSTMs, this results in a simplified architecture with fewer parameters that are easy to train. This paper extends GRUs by introducing the Maxout activation function to learn more expressive memory states.

3. Method: TrajectoryNet

Figure 1 shows the framework of the proposed TrajectoryNet222The source code is available at: The GPS records are divided into segments followed by extraction of point-and-segment-based features. It then discretizes the continuous features, embeds them into another space followed by Maxout GRUs described in Section 3.3 for classification.

Figure 1. Framework of the proposed TrajectoryNet

3.1. Segmentation: From Point-based Features to Segment Features

Many transportation mode detection algorithms divide a trajectory into segments of single-mode trips and assign a single transportation mode to each segment. Zheng et al. (Zheng et al., 2008a) uses walking as the dividing criteria based on the assumption that “people must stop and walk when changing transportation modes” (Zheng et al., 2008a). However, this assumption has one major practical limitation: the absence of the walking segment due to uncertainties in the sampling process could merge a trajectory that has two distinct transportation modes into a single segment. This harms model performance by assigning a single label to this segment that contains two classes. To address this problem, we assign transportation modes based on each discrete GPS sample instead of individual segments. This method enriches the point-based features with segment-based features while preventing the misclassifications incurred by imprecise segmentation.

Various methods can be used for segmenting GPS trajectories, including transition-based method (Zheng et al., 2008a) that uses walking to divide trajectories, clustering-based method (Lee et al., 2007; Soares Júnior et al., 2015) that measures the similarities of sub-trajectories, time-based method (Stenneth et al., 2011) that uses equal time interval in each segment, distance-based method (Liao et al., 2006) that uses equal distance traveled in each segment, bearing-based method (de Souza et al., 2017) that measures changes in bearing orientations and window-based method (Miluzzo et al., 2008) that has the same number of GPS samples in each segment. This paper empirically evaluates the last four methods for their simplicity in implementation.

3.2. Embedding: From Feature Space to Semantic Space

3.2.1. Motivation.

In natural language processing, embedding is the process of converting nominal features, i.e., words, into continuous and distributed vector representations. Compared with local representations, distributed representations have better non-local generalization

(Bengio, 2009), are more efficient and can encode linguistic regularities and semantics.

In the context of transportation mode detection, continuous features—such as speed—are limited by their ability to capture various semantics in different applications. The continuous features can be viewed as observations derived from underlying latent factors that carry distinct semantics in different applications. Take speed as an example, 10km/h is “fast” for a running person but “slow” for a motorist—the same speed can take opposite ends of the spectrum depending on the context. It is not the value of the continuous features that matters, what is more important is how we interpret the meaning. To this end, it is desirable to develop representations that can explain the semantics of the continuous features. In other words, we are interested in converting continuous features into a vector representation that corresponds to their meaning.

3.2.2. The smoothness prior.

One challenge facing the development of embeddings is that, unlike nominal features such as words, there are infinite possible values for continuous features. To address this NP-hard problem (Rousu, 2003), we use the concept of smoothness prior that helps define the way in which we embed continuous features. The task of transportation mode detection exhibits the property that physical attributes (e.g. speed and acceleration) behave continuously, i.e., they generally do not change abruptly in a space or time neighborhood and present some coherence. We introduce a smoothness prior assumption: around the value of a particular continuous attribute, e.g. speed, its semantics are more or less coherent, and changes in transportation modes do not occur all of a sudden.

Theorem 3.1 ().


be a continuous conditional cumulative distribution function of a discrete (categorical) variable

given a continuous random variable

where and are the lower and upper bounds of . For each there exists a finite partition of for such that .


Let , and for define

We first prove . We can prove by contradiction, and by definition , thus we conclude . To prove by contradiction, suppose , by right continuity of the conditional cumulative distribution function , within the neighbourhood of of radius delta there exists , which contradicts with the definition of . Thus, . Next we prove . By definition for . By definition of , we have , which gives . This completes our proof that . ∎

Theorem 3.1 justifies the smoothness prior by stating that there exists a partition, or discretization, of the feature space of random variable , such that the changes in conditional cumulative distribution

within each interval is arbitrarily small. More concretely, in the domain of transportation mode detection, if the cumulative probability of walking

given speed is , there exists a discretization such that within the speed interval defined by this discretization, the cumulative probability of walking is more or less coherent, and changes in the transportation mode do not occur suddenly. The proof of this Theorem is based on Lemma 1.1 (Shaikh, [n. d.]) in the proof of the Glivenko-Cantelli Theorem (Vapnik and Chervonenkis, 2015).

The smoothness prior allows us to embed continuous features by discretizing them into intervals and embed the discretized attributes instead. We only discriminate among different intervals and there is no constraint that different parameterizations are required within the same interval. Discretization has been commonly used in density estimation such as density estimation trees

(Ram and Gray, 2011)

that use piecewise constant function to estimate probability distributions. It has also been used in data mining algorithms, such as C4.5

(Quinlan, 2014)

and Naive Bayes

(Yang and Webb, 2009). From a Bayesian point of view, discretization allows us to use to estimate where is the discretized version of input and is the label. Yang et al. (Yang and Webb, 2009)

shows that “discretization is equivalent to using the true probability density function” in the naive Bayes framework and it is empirically better to use discretization instead of unsafe parametric assumptions of the distribution. Moreover, discretization strengthens parameter estimation through the law of large numbers where more samples are available for each interval compared with infinite values of continuous features. In the context of point-based trajectory classification, it is difficult to “disentangle factors of variations in the data”

(Bengio, 2009) by developing feature compositions of the low-dimensional and dense feature space. Thus it is reasonable to discretize and embed continuous features into vector representations for better classification.

3.2.3. The discretization trick.

(a) Embedding via matrix operation
(b) Embedding via lookup table
Figure 2. Two equivalent embedding implementations

We introduce the “discretization trick” that maps discretized features into the embedding space by matrix multiplication shown in Figure (a)a. The algorithm first converts continuous features into a one-hot vector that indicates the interval in which a feature lies in: the first feature is converted into and the second feature is converted into . The one-hot vectors are then transformed into the embedding space via matrix multiplication , where and . This method is equivalent with the embedding lookup approach, shown in Figure (b)b, that first defines embedding lookup tables and , selects the row vectors ( and ) based on the input values and combines them through element-wise addition. We use the former approach in our experiments as it provides a simple means of embedding by direct matrix multiplication. This embedding is learned through back-propagation.

3.2.4. Discretization strategies.

Meaningful splits are required to provide sufficient predictive power for the semantic embedding. Three discretization strategies are studied in this paper, namely, equal-width binning, Recursive Minimal Entropy Partitioning (RMEP) and fuzzy discretization. Equal-width binning determines the range of each feature and then divides this range with equal-width intervals. This method is unsupervised and straightforward to implement. RMEP (Dougherty et al., 1995) uses Shannon Entropy (Shannon, 2001) to measure the impurity of labels within each partition and recursively partitions each feature using information gain. RMEP improves the supervised predictive power of the resulting intervals. The recursion terminates when a specified number of bins has been reached, or according to Minimal description length principle (Rissanen, 1978). Fuzzy discretization aims to improve results in overlapping data by allowing a continuous value to belong to different intervals in a soft way with some (trapezoidal) membership functions (Roy and Pal, 2003).

3.2.5. Intuitive justifications.

To conclude the discussions on embedding of the feature space, we highlight its beneficial characteristics. Instead of learning layers of neural networks directly on the low-dimensional and heterogeneous feature space with shared parameters for various candidate values of each feature, embedding develops better feature representation by mapping the original features to another space that uses different embedding vectors to represent different values of each feature. This bears some resemblance to the kernel trick. However, adaptive piecewise basis expansions on the feature space is achieved here without explicit design of kernel functions. Whereas the row picture of the embedding matrix represents the process of converting a continuous value into a vector, the column picture of the embedding table can be understood as a type of basis expansion. Table (a)a shows a sample embedding matrix that maps an attribute into a 3-dimensional embedding space. As shown in Figure (b)b, each embedding dimension can be viewed as a basis expansion that detects different patterns from the input space—the first dimension detects values in , the second dimension is a quadratic transformation and the last dimension is an identity approximation. Each basis expansion can be viewed as an expert that specializes in detecting different input-output relationships. The output of the model can be interpreted as a mixture of experts (Jacobs, 1997) that covers different regions of the decision boundary with different nonlinear functions.

(a) Embeddings
(b) Embeddings as basis expansions
Figure 3. Basis expansion: the column picture of the embedding lookup table

3.3. Maxout Gated Recurrent Units

We propose Maxout GRU that applies Maxout activation (Goodfellow et al., 2013) instead of hyperbolic tangent (tanh) when calculating the candidate state

. Maxout activation takes the maximum of a set of linear transformations resulting in an adaptive convex function. It is more expressive than tanh because only two maxout hidden units can approximate any continuous function arbitrarily well

(Goodfellow et al., 2013). It also has better gradient properties. We define Maxout GRU in (2) that takes “expressive” to the next level—in addition to the learned embeddings, we also achieve more flexible memory states. In Equation 2, denotes the index of piecewise linear transformations, is the total number of piecewise transformations and the rest of the notations are the same with Equation 1.


We use this model in conjunction with the discretization method defined in Section 3.2 that maps the input into embeddings such that independent embeddings can be learned to calculate the gates and candidate memory states. In addition to learning the semantic space, this embedding improves bias by reducing correlations between gate units and the candidate states; this is accomplished by decoupling the two lookup matrices and .

3.4. Network Architecture

Figure 4. TrajectoryNet architecture

Figure 4 shows the architecture TrajectoryNet. The first layer learns the embedding space and the second layer learns feature compositions. We use bidirectional GRUs instead of unidirectional GRUs to take account into bidirectional flow of information to achieve better predictive power (Graves et al., 2013).

4. Experiments

4.1. Experiment Settings

4.1.1. Data.

We use the dataset collected by Zheng et al. (Zheng et al., 2008a) and randomly selected 23 people’s trajectories with 2,194,281 GPS records. This dataset provides a large amount of data and a variety of transportation modes. We focus on the four-class classification task and touch upon the seven-class results at the end.

4.1.2. Features.

Two main types of features are used to detect transportation modes: point-based and segment-based features. Point-based features are associated with each individual GPS record and segment-based features are derived from the segments that aim to provide higher orders of information and regularize the noise-sensitive point-based features. More specifically, we calculated the following location-and-user agnostic features using raw GPS records: point-based speed , average speed per segment

and standard deviation of speed per segment

. However, the quality of features calculated from GPS records can be imprecise due to sensor-related reasons (Devogele et al., 2012). To alleviate this uncertainty, we use Hampel filter (Hampel, 1974)

to identify and filter outliers in the feature space.

4.1.3. Network training.

We use truncated backpropagation through time

(Werbos, 1990) to optimize the cross entropy loss with Adam optimizer (Kingma and Ba, 2014) with mini-batches. The threshold of learning rate is 0.01 and we use validation-based early stopping to improve generalization (Bengio, 2012). We use uniform initialization in the range of (Sutskever et al., 2013). We have experimented with different network structures and found that a two-layer structure with 50 hidden nodes in each layer works the best. Each feature is divided into 20 intervals, the embedding dimension is 50 and the Maxout activation consists of 5 pieces of transformations.

4.1.4. Evaluation.

We use Stratified Leave One Batch Out (SLOBO) to evaluate the learned model that divides the data of 23 people into three groups: training, validation and testing set that contain trajectories of 16, 1 and 6 people, respectively. Because the trajectories from different person may contain varying proportions of transportation modes, we select the three subgroups with the objective of getting similar proportions of transportation modes that are representative of the population. The stratification reduces variance in the training process and prevents validation-based early stopping from poor generalization. Because each trajectory has its own intrinsic characteristics, SLOBO also reflects the ability to generalize beyond trajectories of new individuals.

Four methods are selected to measure the model performance: point-based classification accuracy , distance-based accuracy , cross-entropy loss and average F1 score . The descriptions are shown in Table 1.

Measure Description
accuracy based on the number of GPS samples
accuracy based on distances traveled, for comparison with (Zheng et al., 2008a)
insights into the model training process (learning curve)
performance measure when different classes are imbalanced
Table 1. Description of evaluation measures

4.2. Results and Discussion

4.2.1. Comparison with baseline methods.

Table 2

shows the F1 score and accuracy on the test data with distance-based evaluation measures. The most frequent classification errors are between car and bus as well as walk and bus. Overall, TrajectoryNet achieves an encouraging 98% accuracy and outperforms existing baseline methods—including Decision Tree, Support Vector Machine, Naive Bayes, and Conditional Random Field—on the same dataset by a large margin. The improvement in F1 score over the decision tree based framework proposed by Zheng et al.

(Zheng et al., 2008a, b) is 31%, 16%, 22% and 22% for each class. This demonstrates the overall effectiveness of the proposed TrajectoryNet. We further analyze the effect of the individual components of the proposed TrajectoryNet in the rest of this section.

F1 score333F1 scores denoted by “-” are not available in the referred papers. Accuracy
bike car walk bus average
TrajectoryNet (ours) 0.988 0.980 0.972 0.980 0.980 0.979
Decision Tree (Zheng et al., 2008a) 0.675 0.814 0.757 0.748 0.749 0.762
Support Vector Machine (Zheng et al., 2008b) - - - - - 0.462
Naive Bayes (Zheng et al., 2008b) - - - - - 0.523
Conditional Random Field (Zheng et al., 2008b) - - - - - 0.544
Table 2. TrajectoryNet performance on the test data

4.2.2. The effect of embedding.

Figure 5 shows the learning curves of GRUs with and without embedding. All three embedded GRUs have a better (lower) cross entropy loss and converge faster than conventional GRUs: the

embedded GRU achieves in only two epochs is better than the best

that GRU achieves in over 70 epochs. This speedup is because embedding decouples the dependencies between different input values that makes optimization more straightforward. The embedded GRUs are more stable compared with GRU that suffers from exploding gradients in epoch 20 (ReLU) and 55 (tanh). This improvement is due to the fact that even a linear decision boundary in the embedding space can be mapped down to a highly nonlinear function in the original space while being easier to optimize. It reinforces our claim that the representation learned by embedding improves the predictive power of RNNs.

4.2.3. The role of different activation functions.

We also highlight the role of different activation functions in Figure 5

. We find that Maxout activation functions converge better no matter whether embedding is used. This demonstrates that Maxout activation can learn more flexible memory states in GRUs. Note that the fluctuation in the learning curve is a result of Adam stochastic gradient descent with mini-batches. Whereas the tanh and ReLU suffer from exploding gradients without the use of embedding, Maxout activation does not suffer from exploding gradients as it has better gradient properties—piecewise linear.

Figure 5. Learning curves of GRU methods with different activation functions. (R) denotes GRU without embedding and (E) denotes embedded GRU.

4.2.4. The need for segment-based features.

Table 3 shows the classification results when using different combinations of features, and demonstrates that the segment-based features can improve the model performance. This is consistent with our findings about the probability density functions that different features complement each other to provide better separation among various transportation modes. This also demonstrates that the segment-based features can enrich the point-based feature. Note that we used the same network architecture throughout these experiments. Also note that methods marked by in the Tables are significantly better than methods without

at a confidence level of 95% when performing t-test on

, and the differences are not significant among all methods marked by at the 95% confidence level.

0.24 0.91 0.92
0.11 0.97 0.97
0.08 0.98 0.98
Table 3.

Forward feature selection

4.2.5. The effect of segmentation.

The selection of segmentation strategy can be expected to affect the model performance. Table 4 shows results of different segmentation methods. The distance-based method performs the worst because different transportation modes travel at different speeds that result in a varying amount of samples within each segment. This further affects the quality of segment-based features as a result of different sampling complexities in different transportation modes. The rest of the segmentation methods are not statistically different, and the bearing-based segmentation method has the best average performance. Please refer to (de Souza et al., 2017) for further comparison and discussions.

time 0.11 0.95 0.95
distance 0.25 0.86 0.86
bearing 0.08 0.98 0.98
window 0.09 0.97 0.97
Table 4. The effect of segmentation

4.2.6. The effect of discretization.

Table 5 shows the results of different discretization methods. It may seem surprising that equal-width binning works on a par with entropy-based method, but we find equal-width binning robust, easy to implement and easy to train. Fuzzy coding (Roy and Pal, 2003) does not work as well as the previous two methods. Owing to the trapezoidal fuzzy function used in this experiment, the model is forced to learn a weighted sum of two embedding vectors at the same time which makes optimization difficult. This means given the smoothness prior and proper granularities of the partitions, overlapping interval is not a necessity to learn good models. As shown in Table 6, we experimented with various discretization granularities and found that the model was improved significantly when increasing the number of intervals from 10 to 20, but this choice made little difference when the number of intervals is between 20 to 50 for each feature.

width 0.08 0.98 0.98
entropy 0.19 0.95 0.95
fuzzy 0.14 0.96 0.96
Table 5. The effect of discretization strategies
# intervals 10 20 30 40 50
standard error
Table 6. The effect of discretization granularities

4.2.7. Seven-Class Classification.

To further validate the effectiveness of the TrajectoryNet, we undertake a more challenging task: classifying GPS records into seven classes, namely train, car, bus, subway, airplane, and bike. This is a much more challenging task and we achieve 97.3% point-based classification accuracy and 93.0% average F1 score, as shown in Table 7. Compared with the four-class classification task, the reduction in model performance in mainly due to the insufficient amount of training examples for the two classes: subway and airplane.

Target Prediction Recall
train car walk bus subway airplane bike
train 380563 889 853 67 514 1 0 0.994
car 83 66309 261 938 852 0 300 0.965
walk 82 260 139026 2397 784 26 1217 0.967
bus 29 298 2920 121853 76 12 421 0.970
subway 1394 1405 6083 142 24875 10 45 0.733
airplane 4 0 43 23 0 1979 13 0.960
bike 0 0 95 26 14 0 64194 0.998
Precision 0.996 0.959 0.931 0.971 0.917 0.976 0.970
F1 score 0.995 0.962 0.949 0.971 0.815 0.837 0.984
Table 7. Confusion matrix to detect 7 transportation modes ()

4.2.8. Visualizing classification results.

Figure 6 shows the predictions in the test data. The transportation modes, bike, car, walk and bus, are colored in purple, red, yellow, and green, respectively. The misclassified GPS records are colored in black. This figure shows the overall effectiveness of the TrajectoryNet—only a very small fraction of data are misclassified. To further investigate the classification errors incurred from the TrajectoryNet, we highlight three scenarios in the data, labeled as A, B and C in the figure. Scenario A misclassified car into bus and scenario B misclassified bus into car. These are the most common classification errors from the TrajectoryNet; it is challenging to distinguish these two classes as they sometimes manifest similar features. Scenario C are hard to notice from the visualization as they only occur during the transition between two transportation modes. This constitutes another common type of misclassification but only happens in rare occasions implying that the proposed TrajectoryNet is sensitive at detecting transitions between different modes.

Figure 6. Visualizing classification results

5. Conclusion and Future Work

We propose TrajectoryNet—a neural network architecture for point-based trajectory classification to infer real world human transportation modes from GPS traces. To overcome the challenge of capturing the semantics of low-dimensional and heterogeneous feature space imposed by GPS data, we develop a novel representation that embeds the original feature space into another space that can be understood as a form of basis expansion. The embedding can be viewed as a form of basis expansion that improves the predictive power in the original feature space. The embedding can also be viewed as a mixture of experts that specialize in different areas of the decision boundary with different nonlinear functions. We also employ segment-based features to enrich the feature space and use Maxout activations to improve the expressive power of RNNs’ memory states. Our experiments demonstrate that the proposed model achieves substantial improvements over the baseline results with over 98% and 97% classification accuracy when detecting 4 and 7 types of transportation modes.

As for future work, we consider incorporating location-based prior knowledge such as GIS information, developing online classification systems and building user-dependent profiles to further improve this system. We also consider applying the proposed embedding method on other types of low-dimensional and heterogeneous time series data, e.g. Internet of things, to further explore the effectiveness of the proposed embedding method.

The authors acknowledge the support of the NSERC for this research.


  • (1)
  • Ali et al. (2015) Rahman Ali, Muhammad Hameed Siddiqi, and Sungyoung Lee. 2015. Rough set-based approaches for discretization: a compact review. Artificial Intelligence Review 44, 2 (2015), 235–263.
  • Batty et al. (2012) Michael Batty, Kay W Axhausen, Fosca Giannotti, Alexei Pozdnoukhov, Armando Bazzani, Monica Wachowicz, Georgios Ouzounis, and Yuval Portugali. 2012. Smart cities of the future. The European Physical Journal Special Topics 214, 1 (2012), 481–518.
  • Bengio (2009) Yoshua Bengio. 2009. Learning deep architectures for AI.

    Foundations and trends in Machine Learning

    2, 1 (2009), 1–127.
  • Bengio (2012) Yoshua Bengio. 2012. Practical recommendations for gradient-based training of deep architectures. In Neural Networks: Tricks of the Trade. Springer, 437–478.
  • Bengio et al. (1994) Yoshua Bengio, Patrice Simard, and Paolo Frasconi. 1994. Learning long-term dependencies with gradient descent is difficult. IEEE transactions on neural networks 5, 2 (1994), 157–166.
  • Byon et al. (2009) Young-Ji Byon, Baher Abdulhai, and Amer Shalaby. 2009. Real-time transportation mode detection via tracking global positioning system mobile devices. Journal of Intelligent Transportation Systems 13, 4 (2009), 161–170.
  • Cho et al. (2014) Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014).
  • Chung et al. (2014) Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014).
  • de Souza et al. (2016) Erico N de Souza, Kristina Boerder, Stan Matwin, and Boris Worm. 2016. Improving Fishing Pattern Detection from Satellite AIS Using Data Mining and Machine Learning. PloS one 11, 7 (2016), e0158248.
  • de Souza et al. (2017) Erico N de Souza, Johna Latouf, Jiang Xiang, and Stan Matwin. 2017. An Efficient Algorithm to Predict Fishing Activity. To Appear (2017).
  • Devogele et al. (2012) T Devogele, L Etienne, C Ray, and C Claramunt. 2012. Mobility data: Modeling, management, and understanding. (2012).
  • Dodge et al. (2009) Somayeh Dodge, Robert Weibel, and Ehsan Forootan. 2009. Revealing the physics of movement: Comparing the similarity of movement characteristics of different types of moving objects. Computers, Environment and Urban Systems 33, 6 (2009), 419–434.
  • Dougherty et al. (1995) James Dougherty, Ron Kohavi, Mehran Sahami, et al. 1995. Supervised and unsupervised discretization of continuous features. In Machine learning: proceedings of the twelfth international conference, Vol. 12. 194–202.
  • Garcia et al. (2013) Salvador Garcia, Julian Luengo, José Antonio Sáez, Victoria Lopez, and Francisco Herrera. 2013.

    A survey of discretization techniques: Taxonomy and empirical analysis in supervised learning.

    IEEE Transactions on Knowledge and Data Engineering 25, 4 (2013), 734–750.
  • Giannotti et al. (2007) Fosca Giannotti, Mirco Nanni, Fabio Pinelli, and Dino Pedreschi. 2007. Trajectory pattern mining. In Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 330–339.
  • Gonzalez et al. (2010) Paola A Gonzalez, Jeremy S Weinstein, Sean J Barbeau, Miguel A Labrador, Philip L Winters, Nevine L Georggi, and R Perez. 2010. Automating mode detection for travel behaviour analysis by using global positioning systems-enabled mobile phones and neural networks. IET Intelligent Transport Systems 4, 1 (2010), 37–49.
  • Goodfellow et al. (2013) Ian J Goodfellow, David Warde-Farley, Mehdi Mirza, Aaron C Courville, and Yoshua Bengio. 2013. Maxout networks. ICML (3) 28 (2013), 1319–1327.
  • Graves et al. (2013) Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton. 2013. Speech recognition with deep recurrent neural networks. In 2013 IEEE international conference on acoustics, speech and signal processing. IEEE, 6645–6649.
  • Greff et al. (2015) Klaus Greff, Rupesh Kumar Srivastava, Jan Koutník, Bas R Steunebrink, and Jürgen Schmidhuber. 2015. LSTM: A search space odyssey. arXiv preprint arXiv:1503.04069 (2015).
  • Hampel (1974) Frank R Hampel. 1974. The influence curve and its role in robust estimation. J. Amer. Statist. Assoc. 69, 346 (1974), 383–393.
  • Hochreiter (1991) Sepp Hochreiter. 1991. Untersuchungen zu dynamischen neuronalen Netzen. Diploma, Technische Universität München (1991), 91.
  • Hochreiter (1998) Sepp Hochreiter. 1998.

    The vanishing gradient problem during learning recurrent neural nets and problem solutions.

    International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 6, 02 (1998), 107–116.
  • Hu et al. (2016) Baifan Hu, Xiang Jiang, Erico Souza, Ronald Pelot, and Stan Matwin. 2016. Identifying Fishing Activities from AIS Data with Conditional Random Fields. In Proceedings of the 2016 Federated Conference on Computer Science and Information Systems, Vol. 8. IEEE, 47–52.
  • Jacobs (1997) Robert A Jacobs. 1997. Bias/variance analyses of mixtures-of-experts architectures. Neural computation 9, 2 (1997), 369–383.
  • Jiang et al. (2017) Xiang Jiang, Xuan Liu, Erico N de Souza, Baifan Hu, Daniel L. Silver, and Stan Matwin. 2017. Improving Point-based AIS Trajectory Classification with Partition-wise Gated Recurrent Units. In Neural Networks (IJCNN), 2017 International Joint Conference on. IEEE.
  • Kingma and Ba (2014) Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
  • Kisilevich et al. (2010) Slava Kisilevich, Florian Mansmann, Mirco Nanni, and Salvatore Rinzivillo. 2010. Spatio-temporal clustering. Data mining and knowledge discovery handbook (2010), 855–874.
  • Kotsiantis and Kanellopoulos (2006) Sotiris Kotsiantis and Dimitris Kanellopoulos. 2006. Discretization techniques: A recent survey. GESTS International Transactions on Computer Science and Engineering 32, 1 (2006), 47–58.
  • Lee et al. (2007) Jae-Gil Lee, Jiawei Han, and Kyu-Young Whang. 2007. Trajectory clustering: a partition-and-group framework. In Proceedings of the 2007 ACM SIGMOD international conference on Management of data. ACM, 593–604.
  • Liao et al. (2006) Lin Liao, Dieter Fox, and Henry Kautz. 2006. Location-based activity recognition. Advances in Neural Information Processing Systems 18 (2006), 787.
  • Loglisci et al. (2014) Corrado Loglisci, Donato Malerba, and Apostolos N Papadopoulos. 2014. Mining Trajectory Data for Discovering Communities of Moving Objects.. In EDBT/ICDT Workshops. 301–308.
  • Miluzzo et al. (2008) Emiliano Miluzzo, Nicholas D Lane, Kristóf Fodor, Ronald Peterson, Hong Lu, Mirco Musolesi, Shane B Eisenman, Xiao Zheng, and Andrew T Campbell. 2008. Sensing meets mobile social networks: the design, implementation and evaluation of the cenceme application. In Proceedings of the 6th ACM conference on Embedded network sensor systems. ACM, 337–350.
  • Nanni and Pedreschi (2006) Mirco Nanni and Dino Pedreschi. 2006. Time-focused clustering of trajectories of moving objects. Journal of Intelligent Information Systems 27, 3 (2006), 267–289.
  • Quinlan (2014) J Ross Quinlan. 2014. C4. 5: programs for machine learning. Elsevier.
  • Ram and Gray (2011) Parikshit Ram and Alexander G Gray. 2011. Density estimation trees. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 627–635.
  • Rissanen (1978) Jorma Rissanen. 1978. Modeling by shortest data description. Automatica 14, 5 (1978), 465–471.
  • Rousu (2003) Juho Rousu. 2003. Optimal multivariate discretization for naive Bayesian classifiers is NP-hard. (2003).
  • Roy and Pal (2003) Amitava Roy and Sankar K Pal. 2003. Fuzzy discretization of feature space for a rough set classifier. Pattern Recognition Letters 24, 6 (2003), 895–902.
  • Shaikh ([n. d.]) Azeem M. Shaikh. [n. d.]. The Glivenko-Cantelli Theorem. ([n. d.]). Lecture.
  • Shannon (2001) Claude Elwood Shannon. 2001. A mathematical theory of communication. ACM SIGMOBILE Mobile Computing and Communications Review 5, 1 (2001), 3–55.
  • Soares Júnior et al. (2015) Amílcar Soares Júnior, Bruno Neiva Moreno, Valéria Cesário Times, Stan Matwin, and Lucídio dos Anjos Formiga Cabral. 2015. GRASP-UTS: an algorithm for unsupervised trajectory segmentation. International Journal of Geographical Information Science 29, 1 (2015), 46–68.
  • Stenneth et al. (2011) Leon Stenneth, Ouri Wolfson, Philip S Yu, and Bo Xu. 2011. Transportation mode detection using mobile phones and GIS information. In Proceedings of the 19th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. ACM, 54–63.
  • Sutskever et al. (2013) Ilya Sutskever, James Martens, George E Dahl, and Geoffrey E Hinton. 2013.

    On the importance of initialization and momentum in deep learning.

    ICML (3) 28 (2013), 1139–1147.
  • Vapnik and Chervonenkis (2015) Vladimir N Vapnik and A Ya Chervonenkis. 2015. On the uniform convergence of relative frequencies of events to their probabilities. In Measures of Complexity. Springer, 11–30.
  • Werbos (1990) Paul J Werbos. 1990. Backpropagation through time: what it does and how to do it. Proc. IEEE 78, 10 (1990), 1550–1560.
  • Yang and Webb (2009) Ying Yang and Geoffrey I Webb. 2009. Discretization for naive-Bayes learning: managing discretization bias and variance. Machine learning 74, 1 (2009), 39–74.
  • Zhang et al. (2016a) Junbo Zhang, Yu Zheng, and Dekang Qi. 2016a. Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction. arXiv preprint arXiv:1610.00081 (2016).
  • Zhang et al. (2016b) Junbo Zhang, Yu Zheng, Dekang Qi, Ruiyuan Li, and Xiuwen Yi. 2016b. DNN-based prediction model for spatio-temporal data. In Proceedings of the 24th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. ACM, 92.
  • Zheng (2015) Yu Zheng. 2015. Trajectory data mining: an overview. ACM Transactions on Intelligent Systems and Technology (TIST) 6, 3 (2015), 29.
  • Zheng et al. (2014) Yu Zheng, Licia Capra, Ouri Wolfson, and Hai Yang. 2014. Urban computing: concepts, methodologies, and applications. ACM Transactions on Intelligent Systems and Technology (TIST) 5, 3 (2014), 38.
  • Zheng et al. (2008a) Yu Zheng, Quannan Li, Yukun Chen, Xing Xie, and Wei-Ying Ma. 2008a. Understanding mobility based on GPS data. In Proceedings of the 10th international conference on Ubiquitous computing. ACM, 312–321.
  • Zheng et al. (2008b) Yu Zheng, Like Liu, Longhao Wang, and Xing Xie. 2008b. Learning transportation mode from raw gps data for geographic applications on the web. In Proceedings of the 17th international conference on World Wide Web. ACM, 247–256.