There has been a drastic increase in the number of vehicular systems on the market that rely on some degree of automation. These advanced driving assistance systems (ADAS) enhance driver perception and augment the driver’s awareness of the surrounding environment, thereby helping to increase road safety . However, they are not necessarily accepted and utilized by all users, reflecting the perceived trustworthiness of the automation and the skepticism of drivers. This is why user validation of human-centric technology is a pressing issue that is of particular interest in current research.
In high automation, the automated driving system maintains control of the vehicle  and the driver is not required to monitor the road . As a consequence, drivers are free to engage in non-driving tasks . Current prototypes are able to control braking, acceleration and steering but drivers still need to monitor the road, as a response from the driver to a Take Over Request (TOR) is expected.
Today, phones are multi-functional, ubiquitous devices that are capable of connect with other devices in a variety of ways. Modern phones have applications that generate the need for people to check them periodically; so much so that studies show that a large part of drivers use phones while they are driving , . This practice is extremely dangerous in traditional vehicles. However, with the incursion of highly automated vehicles, drivers will be able to safely deviate their attention from the road.
Therefore, the advent of autonomous driving represents an opportunity for increased road safety, as the automation will make superfluous driver intervention in the control of the vehicle . However, the majority of autonomous vehicles currently in operation are used to transport goods or persons within the boundaries of specific industrial sites. In this kind of environment human-machine interaction occurs under relatively controlled conditions and the operators are familiar with the vehicles and the way they function.
In a real road situation the automation contributes to a decrease in driving workload and a consequent reduction in driver situational awareness, which needs to be taken into account when a vehicle control is expected from the driver. Therefore, in high automation driving, Driver State Monitoring Systems (DSMS) play a crucial role in determining whether or not the driver is prepared to take control of the vehicle and to respond to the TOR in the most appropriate manner [8, 9]. An example of DSMS that is employed in some vehicles is a signal that is automatically activated if the driver does not have their hands on the steering wheel for a certain period of time.
In this work we determine the risks of using mobile phones in a field test under real driving conditions while the vehicles’ lateral control is managed by the automation. We examine the impact of the automation on driver behavior and trust by analyzing gaze direction and frequency, as well as the reaction time to a TOR, and formulate the following research question:
Do drivers trust the automated capabilities of the vehicles when they are performing secondary tasks?
by defining the following hypotheses:
Road features (i.e road curves) and secondary tasks do not affect trust in the automation.
The reaction time to take back control of the vehicle does not depend on the kind of secondary task the driver is performing during the automation.
The remainder of the paper is organized as follows: the next section describes related work in the field; section III details the field test. Section IV presents the method used to assess the data collected; section V presents the obtained results; and finally, section VI discusses and concludes the work.
Ii Related Work
A large number of studies have been performed related to the monitoring and assessment of drivers in different situations with the goal of developing systems that optimize autonomous vehicles. An example is the study in  that focused on creating a system that detected the state of drowsiness of the driver, as well as pedestrian and surrounding vehicles, by using the data acquired through the cameras of a smart phone. Based on this information the system could issue a TOR depending on the road situation and the current state of the driver. In the same line of research, the authors in  acquired data from monitoring the driver and the environment to determine the driver’s state of situational awareness using fuzzy logic. In a further work , the architecture of a system that calculated the driver’s state in a highly automated vehicle was presented. The system determined whether it was safe to give the driver control of the vehicle or if the vehicle itself should take care of the corresponding maneuver.
In most studies different external factors were manipulated to determine how they affected the state of the driver and their driving performance. For example, in  and  a variety of images or messages were conveyed to the driver and their cognitive load and stress level were measured afterward using ECG and ECC signals, respectively.
Also related to trust, the works in  and  studied the effects of an adaptive cruise control system on a sample of participants, showing the results that overtrust affected driving performance negatively.
Furthermore  analyzed trust in the automation using simulated vehicles in platoon mode. Results showed that the level of trust depended directly on the efficiency of the vehicle automation.
The existing studies based their data on driver behavior in real or virtual situations using conventional or virtual vehicles. Studies on driver behavior in real road situations that concern automated vehicles are very scarce due to the complexity and the risk factors that they involve. To contribute to the field of research we perform in this study a field test under driving conditions in which drivers’ lateral control is managed by the automation.
Iii Field Test Description
The sample of participants consisted of 15 persons with valid German driving licenses and varying experience with lane keeping systems (LKS). Of the 15 volunteers two participants were discarded because of the noise of their recorded data.
The remaining participants consisted of 5 females and 8 males with an average age of 28.26 years (SD = 9.05) and an average driving experience of 10.15 years (SD = 9.05). Table I provides a visualized summary of the sample characteristics.
|Sample (8 males, 5 females)|
Iii-B Test Procedure
The participants had to drive for a certain time period with an automated lane keeping system activated while performing several predefined secondary tasks. They were instructed to accelerate until they reached 60 km/h, at which time the vehicle’s lane keeping system was activated and the secondary tasks started. The tasks to be performed forced them to deviate their eyes from the road and to remove their hands from the steering wheel.
Prior to the test, they were introduced to the experimental procedure and instructed to keep the system active until a TOR was triggered. This take over request occurred 20 seconds after the driver released their hands from the steering wheel.
A training session was performed before the test so that the participants could get accustomed to the vehicle and its system. During the driving experiment each participant completed three different tasks on the same test track during a total time of 10 minutes.
At the end, each participant completed a post-task questionnaire to determine their situational awareness and self-perceived trust in the system.
Iii-C Secondary Tasks
To assess the impact of the driver assistance system in a real life situation, the secondary tasks were designed to replicate everyday actions that usually occur on the road and are as follows:
Baseline: LKS activated without any task.
Visual: Reading a text (aloud) from a smart phone.
Visual and Manual: Writing a given text on a smart phone.
Both texts were tongue twisters in the native language of the participants (German) in order to slightly increase the mental workload of the task. Figure 1 depicts the test procedure. Figure 2 shows a participant performing one of the secondary tasks while the vehicle was controlled by the lane keeping system.
Iii-D Apparatus and Test Scenario
The test were conducted in a test track using a research vehicle of the Automotive Technology Manufacturers IAV GmbH (Figure 3) that was equipped with a a LKS and other equipment to sense the environment (e.g camera). An internal camera was additionally available to monitor the head pose of the participants. The participants carried out the secondary task on a Samsung S10 plus, which has a 6.4-inch screen using a QWERTY keyboard, instead of the German QWERTZ keyboard, to increase the mental workload of the task.
The scenario chosen to conduct the tests was located in Germany. This track consists of a closed road with two lanes (one for each direction), with a slight curvature of approximately 850 m. For safety reasons, the track was reserved for these tests, without traffic being involved.
Iv Driver Assessment Procedure
In order to test the hypothesis 1 and the level of trust in the automation, we focused on participant gaze direction and frequency, as well as road features since in order to perform the tasks they needed to deviate the eyes from the road and focus on the smart phone. To test the hypothesis 2 we calculated the reaction times necessary to take back control of the vehicle for both types of secondary task performed during the automation.
Iv-a Data Acquisition
To this end the following dependent variables were acquired through the external modules of the vehicle (CAN and camera):
Eyes on road frequency: Defined as the number of times drivers deviated their look from the phone screen to the road.
Eyes on road average time: The average time that the participants look at the road.
Defined as the time it took the participant to put their hands on the steering wheel from the moment in which a TOR was triggered.
The data from the sensors installed for these tests was obtained through a private framework developed by the company IAV.
Iv-B Data Processing and Analysis
To process and further analyze the obtained data it was converted to the format of Robotic Operative System 2 (ROS 2).
The data processing was done semi-automatically by programming two ROS 2 nodes that subscribe to the camera and CAN data independently. The node that was connected to the camera was in charge of showing the recorded images and from these it was observed when the participant deviated their gaze from the road and / or phone. The node was in charge of writing down the times between these two frames, which were pointed out by the observer through the computer keyboard.
In relation to the CAN data, it contained the messages that indicated when the acoustic signal for the TOR occurred and when the participant had taken control of the vehicle. Therefore, to determine the driver’s reaction time, it was only necessary to program a node that would be in charge of measuring the time between these two messages.
To analyze the dependent variables mentioned above, we used the SPSS program where the T-student test was done to determine if there is a relationship between the dependent variables and the secondary task performed.
Furthermore, given that one of the questions asked to the participants was their previous experience with automated capabilities in vehicles, we analyzed a potential correlation between this reported experience and the dependent variables obtained in the experiments.
Finally, we analyzed the relationship between the frequency of diverting the visual attention from the secondary task and features of the road, namely a curve.
Subsequently, we performed a Person Chi Square analysis of the participants post-questionnaire answers in order to obtain a qualitative result of the trust in the systems automated capabilities. The range of the answers varied between 1 (low) and 7 (high).
The results regarding the drivers’ behavioral patterns are illustrated in Table II. It can be observed that there is a significant statistical relationship between the frequency with which drivers diverted their gaze and visual attention from the phone screen to the road and whether they were engaged in the secondary tasks or not (baseline). The eyes on road frequency was higher when they were performing the reading tasks than with the writing task.
|Metric||Baseline||Read secondary task||Write secondary task|
|Eyes on road frequency||4.769||2.087||2.231||1.640||1.461||0.877|
|Eyes on road average time||0.610||0.087||0.641||0.211||0.455||0.154|
|T-test ( =0.05)|
|Metric||Baseline vs read secondary task||Baseline vs write secondary task||Read secondary task vs write secondary task|
|Eyes on road frequency||4.365||0.001**||7.217||0.000**||2.379||0.035*|
|Eyes on road average time||-0.481||0.640||2.600||0.025*||2.793||0.017*|
With regard to the average time that drivers looked at the road and diverted their attention from the phone, there is no statistically significant relationship between the baseline condition and the reading condition. However, there was a statistically significant difference between the baseline condition or the reading condition and the writing condition. Drivers diverted their gaze away from the phone for a shorter period of time when writing on the phone than in the other cases.
Results regarding the reaction time showed that there is a statistical significant relationship between this dependent variable and the task with which the driver was engaged.
The results showed that in most cases the participants needed longer to take control of the vehicle when they were writing on a phone than in the other cases, and at the same time a slower reaction time was obtained when reading than without performing a secondary task under the baseline condition (See Figure 3(a)).
Regarding a correlation between the reported experience with automated systems and the dependent variables obtained in the experiments, Figure 3(b) shows that there was no relationship between the behavior of the drivers in the tests and the experience with the lane keeping system. The correlation coefficients obtained were in any case below 0.13 ().
The results regarding the relationship between a specific road feature such as curves on the road and the visual attention deviation from the phone in Figure 5 show an increase in frequency when drivers approached a curve.
Finally, the level of trust reported by the participants in the experiment can be seen in the Table III. Results showed a statistically significant relationship between the perceived trustworthiness of the automation and the test performed, as the participants reported higher levels of trust in the automation under baseline conditions and less under writing conditions.
Based on the results we can reject the defined hypotheses 1 and 2 and affirm that the type of secondary task and road features affect trust in the automation and that the reaction time to take back the control of the vehicle depends on the kind of secondary task performed during the automation.
|Baseline||Read secondary task||Write secondary task|
|test ( =0.05)|
|Baseline vs Read secondary task||Baseline vs Write secondary task||Read secondary task vs Write secondary task|
Vi Conclusion, Discussion and Future Work
In this paper we presented a study under real field test conditions of the impact of automated driving systems on driver behavior and trust.
We measured visual attention and gaze with respect to secondary tasks and road features, as well as the reaction time to a TOR, and concluded that the secondary task performed by the driver during the automation affected driver behavior and reaction time.
When the drivers were not engaged in a task, their gaze remained on the road. During secondary tasks, the participants’ gaze behavior indicates that they were immersed in their tasks, especially so for the writing task, and therefore trusting of the automation and not worried about a potential failure. It was observed how smart phones make people forget that they are on the road, the task of writing a text causing one to have their eyes off road the longest. This may be because this task requires more concentration to be completed or involves use of one’s hands. However, this objective results did not correspond with the skepticism that resulted from the analysis of the qualitative data that showed that as the task became more complicated, participants reported a lower level of trust in the lane keeping automation.
In more challenging road conditions, the probands care less about the secondary task and trust the system less. Regardless of which secondary task drivers were performing, in most cases they were diverting their attention to the road when the vehicle was approaching a curve, implying a heightened degree of concern about the vehicle system when handling this type of road feature.
Writing on a phone, distracted the drivers from observing their environment, demonstrating how unsafe this task is in traditional vehicles.
The previous experience of some participants with automated vehicular systems did not affect their behavior.
Future work will focus on using specialized sensors to monitor drivers, such as eye tracking devices or sensors to measure stress, as well as external data regarding the driving scenario and conditions.
This work was supported by the Austrian Ministry for Climate Action, Environment, Energy, Mobility, Innovation and Technology (BMK) Endowed Professorship for Sustainable Transport Logistics 4.0.
-  F. Michaeler and C. Olaverri-Monreal, “3D driving simulator with VANET capabilities to assess cooperative systems: 3DSimVanet,” in Intelligent Vehicles Symposium (IV). IEEE, 2017, pp. 999–1004.
-  A. Allamehzadeh and C. Olaverri-Monreal, “Automatic and manual driving paradigms: Cost-efficient mobile application for the assessment of driver inattentiveness and detection of road conditions,” in 2016 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2016, pp. 26–31.
-  T. M. Gasser and D. Westhoff, “Bast-study: Definitions of automation and legal issues in germany,” in Proceedings of the 2012 road vehicle automation workshop. Automation Workshop, 2012.
-  N. Merat, A. H. Jamson, F. C. Lai, and O. Carsten, “Highly automated driving, secondary task performance, and driver state,” Human factors, vol. 54, no. 5, pp. 762–771, 2012.
-  K. M. White, M. K. Hyde, S. P. Walsh, and B. Watson, “Mobile phone use while driving: An investigation of the beliefs influencing drivers’ hands-free and hand-held mobile phone use,” Transportation Research Part F: Traffic Psychology and Behaviour, vol. 13, no. 1, pp. 9–20, 2010.
-  M. E. Gras, M. Cunill, M. J. Sullman, M. Planes, M. Aymerich, and S. Font-Mayolas, “Mobile phone use while driving in a sample of Spanish university workers,” Accident Analysis and Prevention, vol. 39, no. 2, pp. 347–355, mar 2007.
-  C. Olaverri-Monreal, “Autonomous vehicles and smart mobility related technologies,” Infocommunications Journal, vol. 8, no. 2, pp. 17–24, 2016.
-  C. Olaverri-Monreal, S. Kumar, and A. Dìaz-Àlvarez, “Automated driving: Interactive automation control system to enhance situational awareness in conditional automation,” in 2018 IEEE Intelligent Vehicles Symposium (IV), 2018, pp. 1698–1703.
-  J. Çapalar and C. Olaverri-Monreal, “Hypovigilance in limited self-driving automation: Peripheral visual stimulus for a balanced level of automation and cognitive workload,” in 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), 2017, pp. 27–31.
-  A. Allamehzadeh, J. U. de la Parra, A. Hussein, F. Garcia, and C. Olaverri-Monreal, “Cost-efficient driver state and road conditions monitoring system for conditional automation,” in IEEE Intelligent Vehicles Symposium (IV2017). IEEE, 2017, pp. 1497–1502.
-  K. Bylykbashi, E. Qafzezi, M. Ikeda, K. Matsuo, and L. Barolli, “Fuzzy-based Driver Monitoring System (FDMS): Implementation of two intelligent FDMSs and a testbed for safe driving in VANETs,” Future Generation Computer Systems, 2020.
-  J. Goncalves, C. Olaverri-Monreal, and K. Bengler, “Driver Capability Monitoring in Highly Automated Driving: From State to Capability Monitoring,” in IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC, vol. 2015-Octob. Institute of Electrical and Electronics Engineers Inc., oct 2015, pp. 2329–2334.
-  S. Takada, K. Ueda, N. Yamada, Y. Iguchi, Y. Tao, K. Koizumi, and M. Nakao, “Evaluation of Driver’s Cognitive Load when Presented Information on the Windshield using P300 Latency in Eye-fixation Related Potentials,” in 2019 IEEE Intelligent Transportation Systems Conference, ITSC 2019. Institute of Electrical and Electronics Engineers Inc., oct 2019, pp. 4006–4011.
-  Y. Noh, S. Kim, and Y. Yoon, “Evaluation on Diversity of Drivers’ Cognitive Stress Response using EEG and ECG Signals during Real-Traffic Experiment with an Electric Vehicle ∗,” in 2019 IEEE Intelligent Transportation Systems Conference, ITSC 2019. Institute of Electrical and Electronics Engineers Inc., oct 2019, pp. 3987–3992.
-  L. Petersen, L. Robert, J. Yang, and D. Tilbury, “Situational Awareness, Driver’s Trust in Automated Driving Systems and Secondary Task Performance,” SSRN Electronic Journal, mar 2019.
-  J. Beller, M. Heesen, and M. Vollrath, “Improving the driver-automation interaction: An approach using automation uncertainty,” Human Factors, vol. 55, no. 6, pp. 1130–1141, dec 2013.
-  F. M. Verberne, J. Ham, and C. J. Midden, “Trust in smart systems: Sharing driving goals and giving information to increase trustworthiness and acceptability of smart systems in cars,” Human Factors, vol. 54, no. 5, pp. 799–810, oct 2012.
-  H. Xiong, L. N. Boyle, J. Moeckli, B. R. Dow, and T. L. Brown, “Use Patterns Among Early Adopters of Adaptive Cruise Control,” Human Factors: The Journal of the Human Factors and Ergonomics Society, vol. 54, no. 5, pp. 722–733, oct 2012. [Online]. Available: http://journals.sagepub.com/doi/10.1177/0018720811434512