Markov Switching Model for Driver Behavior Prediction: Use cases on Smartphones

08/29/2021 ∙ by Ahmed B. Zaky, et al. ∙ Egypt-Japan University of Science and Technology (E-JUST) Benha University 0

Several intelligent transportation systems focus on studying the various driver behaviors for numerous objectives. This includes the ability to analyze driver actions, sensitivity, distraction, and response time. As the data collection is one of the major concerns for learning and validating different driving situations, we present a driver behavior switching model validated by a low-cost data collection solution using smartphones. The proposed model is validated using a real dataset to predict the driver behavior in short duration periods. A literature survey on motion detection (specifically driving behavior detection using smartphones) is presented. Multiple Markov Switching Variable Auto-Regression (MSVAR) models are implemented to achieve a sophisticated fitting with the collected driver behavior data. This yields more accurate predictions not only for driver behavior but also for the entire driving situation. The performance of the presented models together with a suitable model selection criteria is also presented. The proposed driver behavior prediction framework can potentially be used in accident prediction and driver safety systems.



There are no comments yet.


page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Models for collision avoidance systems, congestion assistants, crash prediction software, etc. have been developed to support a vehicle driver while doing complicated tasks. Accordingly, these systems increase the safety limit for both the drivers and pedestrians. Fully automated autonomous driving systems still need more attention in building models for different driving assistance tasks. Developing a driving behavior model that can be adapted to different driving situations and be able to cover most driving behaviors, is still a challenging task.

Machine learning (ML) is one of the fastest growing areas of science. It has been used in many applications; e.g., traffic signal control Khamis and Gomaa (2014); Khamis et al. (2012), classification, and recognition. ML techniques such as regression models Angkititrakul et al. (2013)

, neural networks (NNs)

Panwai and Dia (2007), and fuzzy systems Ma (2006)

have been used recently in modeling patterns of driving situations. However, such models face the complication of understanding different driving situations (especially the unexpected ones). Driving tasks can be segmented into driving regimes mapped to different driving situations with different response for each driver. A driver usually switches between different behaviors such as car following, lane changing, mobile messaging, sign reading, etc. It is normal to see a driver perform more than one task at the same time, e.g., following a car while switching radio channels or messaging, etc. In this article, we propose a stochastic model that is suitable for detecting and classifying different driving regimes.

Preliminary results of the work presented in this paper have been published in Zaky and Gomaa (2014); Zaky et al. (2015)

. In this paper, the Expectation Maximization (EM) and Markov Chain Monte Carlo (MCMC) are used for estimating the proposed model parameters. Moreover, we calibrate the model for car following driver behavior using our own data set collected by smartphones (as a low cost solution for collecting driving data) plus a naturalistic driving data set presented in

Manstetten et al. (1997). We also present a brief survey for different machine learning models employed for driver behavior and data collection based on smartphones.

The rest part of this paper is organized as follows. Section 2 presents the state of the art literature review. Specifically, we focus on motion detection with use cases on driving behavior detection using smartphones. Also, this section provides the necessary background for the work presented in this paper. This includes detailed description for driver behavior models Section 3

introduces Markov Switching Vector Auto-Regressive model and Bayesian Gipps sampling for model parameters estimation. Section

4 depicts the data collection process using smartphones, the car following dataset, and the adopted driving behavior model. Section 5 presents the results of using both of the data collected using smartphones and the naturalistic driving data. Finally, Section 6 concludes the work presented in this article and provides directions for future research.

2 Related Work

2.1 Driver behavior models

Recently, machine learning approaches have been proposed for driver behavior modeling. Car following is the most popular behavior for evaluating these approaches. Three models are mainly used: Hidden Markov Models (HMM), Gaussian Mixture Models (GMM), and Piece-Wise Auto Regressive Exogenous models (PWARX). These models have achieved remarkable results in simulating driving scenarios. Additionally, these models divide each complex driving pattern into sub-patterns using mixture components. Models introduce different methods for calculating the latent classes, the relationship between the observed variables and each class, the estimation of class parameters, and the number of latent classes.

2.1.1 Hidden Markov models

HMM has been used for driver behavior modeling in different situations such as the model implemented in Dapzol (2005). This model uses sensor data evolution to predict the real current driving situation. The results achieved a prediction accuracy of 80% of driver behavior recognition from the initial driver movements. In Ikeda et al. (2011), the authors presented a collision warning system based on HMM. Traffic models based on HMM have been reviewed in Dapzol (2005) and Sathyanarayana et al. (2008).

2.1.2 Gaussian mixture models

A stochastic driver behavior modeling framework based on GMM is presented in Angkititrakul et al. (2013)

. The model calculates the joint probability distribution for a number of driving signals (following distance, vehicle velocity, brake and gas pedal forces, and vehicle dynamics). The model implements two GMMs as a representation of gas and brake pedals, and their relation with the follower velocity and the gap distance. A main issue with the GMM is the selection of the model components, the authors proposed the use of a Dirichlet process as a non-parametric Bayesian approach that selects the optimal number of model components. The model fits the driving observations for each driver. In addition, a general driver model is implemented based on fitting the observations of several drivers. The authors used different mixtures (4, 8, 16, and 32) for evaluating the model performance.

2.1.3 Piece-wise auto-regressive exogenous models (PWARX)

PWARX have been presented in Sekizawa et al. (2007) and Okuda et al. (2009) to model human driving behavior as a Hybrid Dynamical System (HDS). The proposed approach is switching between simple linear behavior models instead of modeling a non-linear complex model. The driver behavior recognition model introduced in Sekizawa et al. (2007) is a standard HMM extended by embedding an auto-regressive exogenous model (ARX) in each discrete state. The authors introduced a simulation of a collision avoidance system.

In Akita et al. (2008), a car following model classification approach has been introduced based on PWARX as a segmentation approach and a

-means clustering for the input vector. The classification between modes is done using a Support Vector Machine (SVM). The PWARX models have impressive results in modeling driver behavior. However, the PWARX models have two problems

Takeda (2016)

. First, the model can not classify and estimate the behavior simultaneously. Second, it is unable to handle a probabilistic time varying data. The Probability weighted Auto-Regressive model (PrARX) proposed in

Okuda et al. (2013) is an extension to PWARX. The PrARX model addresses these two issues by composing multiple ARX models by a probabilistic weighting function.

The PWARX and GMM models are compared in Takeda (2016) for car following behavior modelling. These models predict a pattern of brake and gas pedal response of the driver as a feedback to the current vehicle velocity and gap distance between vehicles. The results showed that the PWARX approach outperforms GMM in all cases. The prediction of gas pedal behavior was better than the prediction of brake pedal in both models. The results also showed that the operation of gas pedal was smoother than that of the brake pedal. In addition, the gas pedal took longer control time than the pulse-wise shorter period brake pedal.

2.2 Driving behavior detection using smartphones

Motion detection in traffic networks (e.g., anomaly detection, car following driver behavior, etc.) has gained much concern in the last few years. In

Zaky and Gomaa (2014), the authors proposed a Markov regime switching-based model to estimate the driver behavior and extract different driving regimes. The proposed model analyzes a sequence of observations of driving time series data. Trajectory data such as velocity, acceleration, and space gap between the leader and follower drivers were used in model learning. The results showed that by using real car following data sets, the model was able to classify normal car following driving behavior, rare events, and short-time events. In addition, the system was able to determine the switching dynamics among different regimes by applying maximum likelihood estimates and Hamilton filter. Moreover, the proposed model can infer regime specific characteristics such as expected duration, the probability of transferring from one regime to another, switching parameters and driving patterns.

In Ahmed et al. (2007), the authors use two networks; road traffic network and IP network. They recorded images by six cameras over a period of four days. The discrete wavelet transform (DWT) algorithm is used to process the images. The DWT is known for its ability to extract spatially localized frequency information. The authors perform 2D DWT on every image and average the energy of transformation coefficients within each sub-band; a sudden change in the power in the frequency content of the vector of sub-band intensities may represent an anomaly.

The IP network data constitutes a time-series of entropy of four main packet header fields (source IP address, destination IP address, source port number, and destination port number) in each of 11 11 backbone flows in every time step. Anomalies represent changes in the distributions of packets. The authors use two anomaly detection algorithms: One-Class Neighbor Machine (OCNM) and Kernel-based Online Anomaly Detection (KOAD). OCNM uses the k nearest neighbor Euclidean distance as a sparsity measure. KOAD is associated with a kernel function where the features corresponding to the normal traffic measurements are clustered. The region of normality is represented using a relatively small dictionary of approximately linearly independent elements.

In Srivastava et al. (2011), the authors proposed a method to detect anomalies in the trajectory of a vehicle by observing the patterns in its velocity. The authors proposed a look-up table for mapping the trajectory from image co-ordinates to a hypothetical coordinate system in which the axes are selected with respect to the road. In this co-ordinate system, the (axial) velocity is a one dimensional quantity. This is a spatial representation of the velocity which helps localizing the normal behavior definition. The authors estimate the normal modes in the velocity after scaling the velocities by the average speed of the vehicle. The authors detect deviations from the normal velocity for individual sections of the roadway where the normal velocity was modeled using a mixture of Gaussians (obtained by training). The authors use the shape of the trajectory to determine turns and other significant maneuvers, an anomaly is also detected if the velocity of a vehicle falls in a path model which is inconsistent with the shape of its trajectory. The authors characterize the shape of the trajectory using template matching in order to detect turns and other maneuvers. Template matching was performed using sliding windows which does not require pre-segmentation of the trajectory. Thus, it is very suitable for real-time operation.

In Johnson and Trivedi (2011), the authors proposed a novel system that uses dynamic time warping (DTW) and smart phone sensors (accelerometer, gyroscope, magnetometer, GPS, and video) in order to detect and record driving style activities. The proposed system gathers related inter-axial data from multiple sensors into a single classifier. The proposed system utilizes the Euler representation of device attitude (based on the gathered data) to aid the driving activity classification. All the proposed processing is done entirely on the smart phone.

3 Methodology

In this section, we introduce the proposed model framework including model formula, parameter estimation, and characteristics.

3.1 Markov switching vector auto-regressive model

MSVAR Krolzig (2013) is non-linear model which joins the vector auto-regressive models with hidden Markov chain models. The MSVAR model builds a non-linear data model as piece-wise linear model; this is achieved by modeling the process to be linear in each regime. The main objective of such a model is to find the specification of each regime using variables as switches between each regime. Such models use intercept, mean, or both used as switches. For instance, the model presented in Hamilton (1989) utilizes MSVAR with mean switches to study business cycles using the U.S. GDP series. The major difficulty of using the mean as a switching parameter is the estimation of the switching parameters due to their interrelation and the latent variable. On the other side, models that use the intercept as switches require less effort in estimation, it can be estimated using Monte Carlo methods.

The model introduced is based on multivariate time-series consisting of observations, where represents an N-dimensional vector and a stochastic process that depends on a latent discrete stochastic process, , having discrete state-space with state variable which indicates the dominant regime at time . The reduced form of the model is presented in Krolzig (2013) and is known by MSIAH-VAR(p) as in (1). The reduced model uses three types of switches: intercept , regression coefficient

, and co-variance matrix




The state variable is evolved over time as a discrete time, discrete space Markov process, assuming for regimes. Let represents the number of appropriate regimes, so that . The conditional probability density of the observed time series vector is given by (2) where is the VAR model parameter vector for regime and are the observations from time to time .


The stochastic transition of states is determined by a Markov transition matrix which determines the dynamics of the switching process where = is the probability of switching from state to state and .

The model parameter state vector is defined as follows:


This model parameter state vector can be estimated by maximum likelihood as in Eq. (4). The estimation process is performed through the Expectation Maximization (EM) algorithm that is presented in Hamilton and Susmel (1994). EM iteratively calculates the next step of the state vector given the previous observation and the previous state vector using the log-likelihood function of the data. The algorithm proceeds in two steps: expectation and maximization steps. The expectation step uses the parameter estimated from the previous maximization step to compute both filtered probability vectors.

The likelihood is proportional to the probability of observing the data given the estimated parameter. The minimization of the log-likelihood in Eq. (4) can be used as an objective for parameter estimation and for comparing the different model fitting schemes.


There are three main terminologies for gaining information about different driving regimes: driving regime inference, regime classification, and regime expected duration.

3.1.1 Regime inference

The objective of the regime inference process is the identification of the latent regime variable from the observations . This process requires two main steps: filtering and smoothing.

Filtered probability estimates The filter step aims at estimating which represents the probability of the unobserved state vector . The probability of being under regime at time given the model parameters is given by Eq. (5):


where is the sequence of observations over time given by:


and is the population parameter vector which is given by:


with the constraints that and .

Smoothing The filter step will generate estimates for state where using observations up to time . The smoothing task improves regime inference by taking into consideration the future observation . The smoothed probability is defined as . The smoothed algorithm is a backward filter that starts from the last observation point . The smoothing algorithm starts by calculating the smoothed probability of the last observation point , and then iterates backward to ; the algorithm steps are as follows:

  1. The smoothed probability of the last observation point:

  2. According to the Markovian assumption, the depends only on :

  3. The calculation of the smoothed probability is done by using the last term of the previous iteration where

  4. The recursion is initialized with the final filtered probability vector . The following equation shows how the future observation improves the inference of the unobserved state .

3.1.2 Regimes classification

The classification of regimes begins by assigning each observation to a regime . The classification is achieved by mapping each observation to the winning regime with the highest smoothed probability as in Eq. (6).


The smoothed regime probabilities are calculated using the dataset. Then each observation is assigned to the highest regime filtered probability.

3.1.3 Regime expected duration

The expected length of stay in a specific regime (state ) can be derived from the regime transition matrix; this is achieved by using the probability of staying in the same regime . Let be the time period in which the system stays at regime . Equation (7) is the probability to stay time period in regime . The expected duration can be specified by the formula presented in Eq. (8), and according to the formula the expected duration depends only on the transition probability of the same regime, so the expected duration remains constant over time and the higher the transition probability, the longer the stay for the regime.


3.1.4 Gipps bayesian parameter estimation

The estimation of the MSVAR model is a difficult task. Parameter estimation can be done easily for Bayesian models having known closed form posterior distributions. The main objective of our proposed model is to adopt the MSVAR in driver behavior modeling where we cannot determine the model parameters of prior distributions directly. MCMC can be used for finding the posterior distribution for the model parameters. This is attributed to the ability of MCMC to generate samples from the posterior distribution. The estimation of the proposed MSVAR model has been implemented based on Gipps sampler for posterior distribution sampling.

The Gipps sampler is a popular and efficient MCMC sampling algorithm 4. The Gipps sampler is like the Metropolis Hastings (MH) component-wise implementation in sampling each dimension. However, instead of sampling each dimension from an independent proposed distribution, Gipps samples from variable full conditional distribution . The algorithm accepts all drawn samples, thus Gipps has lower computation requirements and converges faster Gipps (1981). Like the component-wise implementation, the Gipps algorithm step-samples through each variable while the other variables are fixed.

If the target conditional distribution is belonging to standard distributions, then the sampling can directly be done from these distributions, otherwise Metropolis–Hastings algorithm can be used for sampling the target distribution. The Bayesian parameter estimation approach assumes that both of the regime and the model parameters

are random variables. The Gipps sampler has been used in the inference of the state-space models, even it has been used for classifying the states without estimating the model parameters. The Gipps sampler can be used for sampling the parameters of the posterior distributions.

The sampler draws samples from the latent states and samples the model parameters from the full conditional distribution. The sampler starts with sampling from where is the observed data, then sampling from . The prior specification of the state-space sampling for known number of states is followed (as presented in Richardson and Green (1997)) where the standard distribution families selected for model parameters are implemented and the model parameters are conditionally independent. The parameters prior distribution are estimated as follows:

  • The joint transition probabilities use independent Dirichlet distribution prior for each state.

  • Each regression coefficient mean has independent Gaussian prior.

  • Each regression coefficient standard deviation has gamma prior.

The presented prior specification has full state conditional distribution which follows a Dirichlet distribution as in Eq. (9) where is an indicator function for indicating the current state.


The joint transition probabilities for each state has full conditional distribution which follows a Dirichlet distribution as in Eq. (10) where is the number of transitions from state to state .


The Gipps sampler iterates on two steps. Step (a) for updating the parameters and Step (b) for Markov chain revised as follows:

  • Step a

    • Update the mean by sampling from the Gaussian prior.

    • Update the standard deviation gamma prior.

    • Update the transition probabilities for each state independently by sampling from the proposed distribution in Eq. (10).

    • Update the from the proposed distribution in Eq. (9).

  • Step b

    • Update the filtered probability .

    • Update the transition probabilities

4 Experimentation

4.1 Car following data set

The Robert Bosch GmbH Research Group Manstetten et al. (1997) floating car dataset (FCD) is used to validate our model. This dataset represents a car following behavior of vehicle speed under stop-and-go traffic conditions during an afternoon peak on a single lane in Stuttgart, Germany. A car with a frontal radar sensor based on a Doppler ultrasound is used to measure the relative speed and distance between a leader and a follower drivers. The used datasets are sampled at 100 ms with duration of 250, 400, and 300 seconds. Data set 1 gap distance, speed, and acceleration are shown in Fig. 1.

Figure 1: Dataset_1 gap distance, velocity difference, and acceleration.

These datasets have complex situations in daily urban traffic with lots of acceleration and deceleration periods. Due to the existence of traffic lights in the recorded scenario, there are some standstill periods. The velocity varies in the range between 0 km/h and 60 km/h. These datasets are used in modeling, evaluating, and calibrating car following models such as the Intelligent Driver Model (IDM) in Treiber and Kesting (2013), the neural network models Panwai and Dia (2007), and the state-space models Hoogendoorn et al. (2007).

4.2 Data collection

The driving data collection is a complex task. Most vehicle data collection experiments consist of high quality recording of driver’s behavior. Lots of sensors have to be equipped within the vehicle equipment in order to record the various driver behavior signals. Sensors such as microphones, video camera, steering wheel angle, gas pedal, brake pedal, GPS, speed, acceleration, and heart rate can be used according to the objective behind the study.

The collection of driving behavior with this procedure has a high cost. Thus, a low cost solution using smartphones to collect car following behavior data has been introduced. Sensor data from both follower and leader vehicles is highly beneficial and accordingly used to fit the proposed model (using smartphones iPhone 6 and 6 plus). We converted the GPS latitude and longitude to the actual distance between the two vehicles considering the spherical shape of the earth using the Haversine formula Nitin R.Chopde1 (2013).

The driving experiment has the following characteristics:

  • There are follower and leader drivers with predefined set of behaviors.

  • Every driver has one smartphone in his vehicle.

  • Every smartphone has SensorLog application running (an application for logging sensors data).

  • All sensors are being logged; e.g., GPS, accelerometer, gyroscope, compass (location heading).

  • Sampling rate of GPS is 1 Hz (1 sample per second).

  • Sampling rate of acceleromoter is 100 Hz.

  • We depend on GPS samples for localization. We obtain distances, velocities, and accelerations from GPS readings. The accelerometer does not readily provide acceleration data since it is relative to free fall.

  • Velocity is calculated by SensorLog (using 2 consecutive GPS readings).

  • Acceleration is calculated from the velocity difference.

  • We need to estimate, and accordingly classify, the follower velocity based on follower acceleration, velocity difference, and gap distance.

4.3 Adopted driving behavior model

MSVAR models presented in Eq. (1) are adopted using driving signals that represent the car following behaviors. The signals used are presented in Eq. (11) where the observation vector consists of four observation signals . These signals represent the car following driver behavior where is the follower velocity, is the follower acceleration, is the difference in velocity between the leader and follower, and is the gap distance between the leader and follower. The prediction of the model has interval forecasts which can be evaluated using the conditional mean and the mean square prediction error (MSPE).

The objective is to find the conditional density of given the model parameters and the previous observation . The prediction density given in Eq. (12) is a mixture of normals, and is the probability of each predicted regime.


5 Results and Discussion

5.1 Results of data collected using smartphones

The data collected from the smartphones have been fitted using the Markov Regime Switching model presented in Eq. (14), where the driving signals selected as features are: follower velocity , acceleration , velocity difference , and gap distance . The model is introduced in our previous work Zaky and Gomaa (2014) with the model ability to classify the different car following driving regimes.


The estimated model parameters and are presented in Table 2 for each regime, where the log-likelihood approach is used. Markov transition matrix estimated, where is the probability of moving from driving regime to driving regime :

Table 1 presents the observed information of each driving regime in the dataset. We can observe the expected duration in which a driver can stay in each regime based on Eq. (8). This means that the driver will stay for a time around the expected duration time driving in that regime. Other characteristics of regimes are shown such as the number of occurrence which counts the samples inside each regime, the number of observations that belongs to each regime, and the percentage of driving under each regime overall time.

Regime Characteristics
duration (ms)
Occurrence Observations Percentage
Regime 1 17.50 4 276 67%
Regime 2 6.79 8 42 10%
Regime 3 5.34 10 34 8.25%
Regime 4 1.91 11 60 14.5%
Table 1: Driving regimes contained in the adopted dataset.
Parameter Regime 1 Regime 2 Regime 3 Regime 4
1.3346 1.2511 1.6234 1.9769
1.2163 -0.4069 0.0671 -1.5901
1.5228 1.2641 0.3858 0.9247
0.6563 0.5074 0.2625 0.3031
Table 2: Estimates of car following Markov regime switching model parameters.

We have conducted several trials and due to the noise of the sensors specially the GPS (which has up to 4 meters error with a sampling rate of 1 sample per second), our experiment is limited by driving in only three car following situations: acceleration, braking, and normal following. A manual tagging for the car following situations is done. The interpretation of the results is as following:

  • Tag 0: is the acceleration behavior which the model classifies as Regime 2.

  • Tag 2: is the braking behavior which the model classifies as Regime 3.

  • Tag 4: is the stable following behavior which the model classifies as Regime 1.

The results comply with the labels taken by the follower driver that is out of 412 records (seconds):

  • 320 records are tagged as label 4 (stable following). The model classifies 276 observations with mis-classification of 44 observations, i.e., accuracy of 86.25%.

  • 49 records are tagged as label 0 (acceleration). The model classifies 42 observations with mis-classification of 7 observations, i.e., accuracy of 85.42%.

  • 43 records are tagged as label 2 (braking). The model classifies 34 observations with mis-classification of 9 observations, i.e., accuracy of 79%.

5.2 Results of the naturalistic driving data

We have implemented different MSVAR models with different regimes, varying from 2 to 5 regimes and different time lag varying from 2 to 5 ms. The maximum log-likelihood values are shown in Table 3 for different lags and regimes. As shown in Table 3 the best model fit is for lag 1 having 5 regimes with the maximum likelihood value. The using of the lag can be useful for modeling the driver sensitivity factor which is presented in all GM models Rakha et al. (2009). The following driver responds to the leading driver action by acceleration or deceleration depending on the driver perception, reaction time, and driver sensitivity factors. An average reaction time is estimated in a range between 1.0 to 2.2 seconds, and an average driver sensitivity factor of 0.37 second as introduced in May (1990).

RegimeLag 1 2 3 4 5
2 10036.68 10642.67 10739.58 11057.45 11090.25
3 9908.593 10869.07 10992 10837.91 10889.22
4 10151.84 10400.08 10981.64 10515.56 10499.64
5 10377.99 10244 10359.58 10341.47 10005.32
6 10191.81 9348.61 9985.96 10054.52 10119.44
Table 3: MSVAR log likelihood values.

Most VAR models are estimated using symmetric lags, i.e., the same lag length is used for all variables of the model. The model lag length can be determined based on a specific selection criteria. Model selection criteria such as Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), and Hannan-Quinn Information Criterion (HQC) are used. The selection of an inappropriate lag length may affect the model performance and fitting.

The results of applying each criteria on the adopted dataset are shown in Fig. 2. This figure shows the results for different lags between 1 and 60 ms. This helps us to exploit the history of the driving situation in the model. The dataset is sampled at 10 Hz, thus 1 lag means 0.1 second while 60 lag means 6 seconds. The minimum values of the three selection criteria falling between 5 and 13 (0.5 and 1.3 seconds) can be observed which represent the different reaction times of the driver.

(a) AIC values for lag between 1 and 60 ms.
(b) BIC values for lag between 1 and 20 ms.
(c) HQ values for lag between 1 and 20 ms.
Figure 2: MSVAR(P) model selection criteria.

The MSVAR model presented in Table 3 is used for fitting the naturalistic driving data. The presented results are for the dataset which contains 2529 observations. As shown in the table, the best fitting is for the model with a lag of 5 and 2 regimes. The lag selection criteria described above uses 2520 observations, and the remaining 9 observations have been used for models forecasting evaluation.

The forecasting results based on Eq. (12) represent a comparison of the prediction performance of the two selected models. The highest max log-likelihood model is selected. The first model has 1 lag and 5 regimes Model I (p=1,r=5), while the second model has 5 lags and 2 regimes Model II (p=5,r=2). Table 4 shows a comparison between the Mean Square Error (MSE) of the observation vector for the two models. The error is calculated between the dataset observed values and the predicted values for 9 samples (0.9 second). As shown, Model II has a lower MSE for all observation vector elements.

I (p=1,R=5) 0.1788184 0.013395644 0.05679842 0.02309714
II (p=5,R=2) 0.1481768 0.010264159 0.05339007 0.01554496
Table 4: Mean square error of the two MSVAR models.

Fig. 3 presents the prediction of the two models for each point. The red points represent the naturalistic real driving data samples, blue points represent Model I, and green points represent Model II. Both models are accurate for the first forecasting steps, afterwards the models start deviating due to the accumulated error of the forecasting process (as shown in the velocity and velocity difference figures). The models are able to predict not only the driver behavior represented by its observation (velocity and acceleration), but also the entire driving situation represented by the relation with the leading vehicle observation (gap distance and velocity difference).

(a) DVP
(b) HP
(c) VP
Figure 3: MSVAR model predicted values of car following observations.

5.3 MSVAR versus PrARX

The proposed MSVAR framework, as a switching linear regression model for driving behavior, has features listed in Table


. The table also lists the features of PrARX as a switching framework based on linear regression. Both frameworks are able to provide behavior/mode segmentation extracted from driving signals. PrARX uses K-means clustering and the proposed framework uses a probabilistic classifier based on selecting the maximum state filtering value for each observation. The probabilistic classification approach is able to identify the membership probability of a new observation to each state (finds the best state that represents the observation with the highest probability). The advantage of probabilistic classifier over non-probabilistic one is that the former behaves like a confidence weighted classifier which helps avoiding error propagation. The classifier adds value for the driving behavior problem by allowing a smooth transition between each regime allowing a mixed mode representation to understand the current behaviors that the driver may behave.

PrARX Proposed MSVAR framework
Behavior (Mode) segmentation Extension to k-means clustering Probabilistic classification
Learning process Cannot simultaneously classify and estimate Classify and estimate in a single recursive process
Parameter estimation method Steepest descent ML (Hamilton) and Bayesian (Gipps Sampling)
Computational cost High due to the processing of the classification and estimation independently MCMC minimize computational time
Assumptions Human driving behavior does not exhibit an abrupt change Abnormal behavior detected and can capture short time events - behavior and switching processes are stationary
Mixed mode Can Can
Table 5: Main features of PrARX model and the proposed MSVAR framework

The learning process of PrARX has two stages; behavior classification and parameters estimation. PrARX framework cannot simultaneously classify and estimate. The proposed framework classifies and estimates in a recursive process by using Hamilton filter for classification and maximum likelihood for parameter estimation. Computational cost of PrARX is higher due to the processing of classification and estimation independently. For the proposed model, a Bayesian parameter estimation based on MCMC minimizes computational time over Expectation Maximization (EM). As a future work, performance measurements for the framework evaluation will be conducted.

PrARX has a major assumption for relaxing the problem of parameter estimation. The assumption is that human driving behavior does not exhibit an abrupt change which allows the framework to use the parameters of the previous behavior as initial values for the next one. The proposed MSVAR framework does not have this assumption as the model parameter estimation process generates independent parameters for each regime, which allows the framework to handle abnormal behavior. The proposed MSVAR framework has an assumption that the driver behavior is stationary, so the stochastic process representing each regime and the switching process (transition matrix) have the same parameters over time. Releasing this assumption requires a variable time transition probabilities and more complex stochastic models for driver behavior modeling such as Gaussian Mixture Models (GMM).

6 Conclusions and Future Work

The prediction of driving behavior based on multiple Markov Switching Variable Auto-Regression (MSVAR) is introduced. More than one model are implemented with different parameters (lag and regime) and with different evaluation criteria (AIC, BIC, HQC). The best fitted models are selected for the prediction process. Additionally, the model is capable of fitting driving data and data segmentation into regimes by estimating the different driving behavior change points. One limitation is the long calibration time of the model parameters. This is attributed to the learning of each model (depending on the model configuration the learning takes up to 3 days). The best fitted model is achieved at lag of 13 (1.3 s). We have implemented models with lags down to 5 (0.5 seconds). The computational efficiency of the prediction is reasonable; it takes only few seconds, however, it needs more adaptation to be more accurate. We also presented a low-cost data collection solution using smartphones validated with another naturalistic driving data set for predicting the driver behavior for short periods of time. The proposed driver behavior detection model can potentially be used in systems such as accident prediction and driver safety.

7 Acknowledgements

This work is mainly supported by the Ministry of Higher Education (MoHE) of Egypt through PhD fellowship awarded to Dr. Ahmed Zaky. This work is supported in part by the Science and Technology Development Fund (STDF), Project ID 12602 ”Integrated Traffic Management and Control System”, and by E-JUST Research Fellowship awarded to Dr. Mohamed A. Khamis.


  • T. Ahmed, B. Oreshkin, and M. Coates (2007) Machine learning approaches to network anomaly detection. In Proceedings of the 2nd USENIX workshop on Tackling computer systems problems with machine learning techniques, Cited by: §2.2.
  • T. Akita, T. Suzuki, S. Hayakawa, and S. Inagaki (2008) Analysis and synthesis of driving behavior based on mode segmentation. In Control, Automation and Systems, 2008. ICCAS 2008. International Conference on, pp. 2884–2889. Cited by: §2.1.3.
  • P. Angkititrakul, C. Miyajima, and K. Takeda (2013) Stochastic mixture modeling of driving behavior during car following. Journal of information and communication convergence engineering 11 (2), pp. 95–102. Cited by: §1, §2.1.2.
  • [4] (2013) Bayesian data analysis third edition.. Cited by: §3.1.4.
  • N. Dapzol (2005)

    Driver’s behaviour modelling using the hidden markov model formalism

    In ECTRI Young researchers seminar, The Hague, the Netherlands, Vol. 2, pp. 2–1. Cited by: §2.1.1.
  • P. G. Gipps (1981) A behavioural car-following model for computer simulation. Transportation Research Part B: Methodological 15 (2), pp. 105–111. Cited by: §3.1.4.
  • J. D. Hamilton and R. Susmel (1994) Autoregressive conditional heteroskedasticity and changes in regime. Journal of Econometrics 64 (1), pp. 307–333. Cited by: §3.1.
  • J. D. Hamilton (1989) A new approach to the economic analysis of nonstationary time series and the business cycle. Econometrica: Journal of the Econometric Society, pp. 357–384. Cited by: §3.1.
  • S. P. Hoogendoorn, S. Ossen, and M. Schreuder (2007) Adaptive car-following behavior identification by unscented particle filtering. In Transportation Research Board 86th Annual Meeting, Cited by: §4.1.
  • K. Ikeda, H. Mima, Y. Inoue, T. Shibata, N. Fukaya, K. Hitomi, and T. Bando (2011) An Adaptive Rear-End Collision Warning System for Drivers That Estimates Driving Phase and Selects Training Data. Transactions of the Institute of Systems, Control and Information Engineers 24, pp. 193–199. External Links: Document Cited by: §2.1.1.
  • D. A. Johnson and M. M. Trivedi (2011) Driving style recognition using a smartphone as a sensor platform. In Proceedings of the 14th International IEEE Conference on Intelligent Transportation Systems, Washington, DC. Cited by: §2.2.
  • M. A. Khamis, W. Gomaa, A. El-Mahdy, and A. Shoukry (2012)

    Adaptive traffic control system based on bayesian probability interpretation

    In Proc. IEEE 2012 Japan-Egypt Conference on Electronics, Communications and Computers (JEC-ECC 2012), Alexandria, Egypt, pp. 151–156. Cited by: §1.
  • M. A. Khamis and W. Gomaa (2014)

    Adaptive multi-objective reinforcement learning with hybrid exploration for traffic signal control based on cooperative multi-agent framework


    Journal of Engineering Applications of Artificial Intelligence

    29, pp. 134–151.
    Cited by: §1.
  • H. Krolzig (2013)

    Markov-switching vector autoregressions: modelling, statistical inference, and application to business cycle analysis

    Vol. 454, Springer Science & Business Media. Cited by: §3.1, §3.1.
  • X. Ma (2006) A neural-fuzzy framework for modeling car-following behavior. In Systems, Man and Cybernetics, 2006. SMC’06. IEEE International Conference on, Vol. 2, pp. 1178–1183. Cited by: §1.
  • D. Manstetten, W. Krautter, and T. Schwab (1997) Traffic simulation supporting urban control system development. In Mobility For Everyone. 4Th World Congress On Intelligent Transport Systems, 21-24 October 1997, Berlin.(Paper No. 2055), Cited by: §1, §4.1.
  • A. D. May (1990) Traffic flow fundamentals. Prentice-Hall. Cited by: §5.2.
  • Mr. M. K. N. Nitin R.Chopde1 (2013) Landmark based shortest path detection by using a* and haversine formula. International Journal of Innovative Research in Computer and Communication Engineering 92, pp. 298–302. Cited by: §4.2.
  • H. Okuda, N. Ikami, T. Suzuki, Y. Tazaki, and K. Takeda (2013) Modeling and analysis of driving behavior based on a probability-weighted arx model. Intelligent Transportation Systems, IEEE Transactions on 14 (1), pp. 98–112. Cited by: §2.1.3.
  • H. Okuda, T. Suzuki, A. Nakano, S. Inagaki, and S. Hayakawa (2009) Multi-hierarchical modeling of driving behavior using dynamics-based mode segmentation. IEICE transactions on fundamentals of electronics, communications and computer sciences 92 (11), pp. 2763–2771. Cited by: §2.1.3.
  • S. Panwai and H. Dia (2007) Neural agent car-following models. Intelligent Transportation Systems, IEEE Transactions on 8 (1), pp. 60–70. Cited by: §1, §4.1.
  • H. Rakha, P. Pasumarthy, and S. Adjerid (2009) A simplified behavioral vehicle longitudinal motion model. Transportation letters 1 (2), pp. 95–110. Cited by: §5.2.
  • S. Richardson and P. J. Green (1997) On bayesian analysis of mixtures with an unknown number of components. Journal of the Royal Statistical Society. Series B (Methodological), pp. 731–792. Cited by: §3.1.4.
  • A. Sathyanarayana, P. Boyraz, and J. H. Hansen (2008) Driver behavior analysis and route recognition by hidden markov models. In Vehicular Electronics and Safety, 2008. ICVES 2008. IEEE International Conference on, pp. 276–281. Cited by: §2.1.1.
  • S. Sekizawa, S. Inagaki, T. Suzuki, S. Hayakawa, N. Tsuchida, T. Tsuda, and H. Fujinami (2007) Modeling and recognition of driving behavior based on stochastic switched arx model. Intelligent Transportation Systems, IEEE Transactions on 8 (4), pp. 593–606. Cited by: §2.1.3.
  • S. Srivastava, K. K. Ng, and E. J. Delp (2011) Co-ordinate mapping and analysis of vehicle trajectory for anomaly detection. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME), Cited by: §2.2.
  • K. Takeda (2016) Modeling and detecting excessive trust from behavior signals: overview of research project and results. In Human-Harmonized Information Technology, Volume 1, pp. 57–75. Cited by: §2.1.3, §2.1.3.
  • M. Treiber and A. Kesting (2013) Microscopic calibration and validation of car-following models–a systematic approach. Procedia-Social and Behavioral Sciences 80, pp. 922–939. Cited by: §4.1.
  • A. B. Zaky, W. Gomaa, and M. A. Khamis (2015) Car following markov regime classification and calibration. In Proceedings of the IEEE 14th International Conference on Machine Learning and Applications (ICMLA 2015), Miami, Florida, USA. Cited by: §1.
  • A. B. Zaky and W. Gomaa (2014) Car following regime taxonomy based on markov switching. In Proceedings of the IEEE 17th International Conference on Intelligent Transportation Systems (ITSC 2014), Qingdao, China, pp. 1329–1334. Cited by: §1, §2.2, §5.1.