Light Virtual Reality systems for the training of conditionally automated vehicle drivers

03/13/2018 ∙ by Daniele Sportillo, et al. ∙ MINES ParisTech 0

In conditionally automated vehicles, drivers can engage in secondary activities while traveling to their destination. However, drivers are required to appropriately respond, in a limited amount of time, to a take-over request when the system reaches its functional boundaries. In this context, Virtual Reality systems represent a promising training and learning tool to properly familiarize drivers with the automated vehicle and allow them to interact with the novel equipment involved. In this study, the effectiveness of an Head-Mounted display (HMD)-based training program for acquiring interaction skills in automated cars was compared to a user manual and a fixed-base simulator. Results show that the training system affects the take-over performances evaluated in a test drive in a high-end driving simulator. Moreover, self-reported measures indicate that the HMD-based training is preferred with respect to the other systems.



There are no comments yet.


page 2

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Methods

This study contained two parts: training and test drive. The aim of the training was to introduce the principles of the Level 3 Automated Driving System (ADS)-equipped vehicle, present the novel human-machine interfaces, and describe the actions to perform in order to appropriately respond to unplanned requests to intervene. A between-subject study with 60 participants was designed in order to compare a light Virtual Reality systems to a user manual and a fixed-base driving simulator in terms of training effectiveness evaluated through a test drive which required the application of knowledge and skills acquired during the training.

1.1 The Training

The aim of the training was to teach drivers how to interact with automated cars in three situations: the manual mode, automated mode and the take-over request. To do so, the training introduced the participants to the HMI for each situation, the actions they were free to perform during the automated driving and the best practice to respond to a take-over request. For all the participants, the training program started with an introductory video that briefly presented the main functionalities of a Level 3 ADS-equipped car.

In the study three different training systems were compared. The first one was a user manual (UM) consisting of a slide presentation displayed on a 13.3" screen of a laptop computer. The participants were asked to carefully read each of the 8 slides and to proceed when they felt ready, without a time limit. The slides used text and images to present the actions to be performed during the manual driving, the automated driving and the take-over requests.

The second system was a fixed-base simulator (FB) consisting of an actual car cockpit including a driving seat, a dashboard, a force-feedback steering wheel and a set of pedals. A 9.7" tablet used by the driver to perform the secondary activity was placed in the center console. To display the virtual environment a 65" plasma screen was positioned behind the cockpit at 1.5m from the driver.

The third system was a Light Virtual Reality (LVR) system including an HMD as a display system, and a racing wheel as driving system. Spatial sound was presented via headphones. To have a spatial correspondence between the real steering wheel and the virtual one, the steering wheel inside the virtual car was a 3D model of the real racing wheel with which the participants were interacting. Moreover, the position and the movements of the virtual model corresponded to the real one, allowing for co-located manipulation.

1.1.1 The Virtual Learning Environment

For the training using the LVR system and the fixed-base driving simulator, a step-by-step tutorial was developed in the form of a Virtual Learning Environment (VLE) (Figure 1). The task of the participants consisted of interactions with the car following the instruction of a virtual vocal assistant. The messages announced by the assistant were also displayed on a yellow panel in front of the trainee. The driving scenario was a straight 2-lane road delimited by guardrails with no traffic. Before the driving scenario, an acclimatization virtual environment was proposed to the participants to help them locate and identify the controls of the car. This training also included a secondary activity (a video) that required the use of a tablet to distract the human driver from the driving task during the automated driving. The participants were asked, but not forced, to look at the tablet. The video was automatically played when the automated system was enabled and paused during the manual driving and the TORs.

1.2 Test Drive

Figure 1: Illustration of a user with HMD immersed in the Virtual Leaning Environment

After the training, the participant performed a test drive designed to evaluate their performance in a more realistic driving scenario. The system used for this purpose was a high-end driving simulator consisting of the front part of a real car surrounded by a panoramic display. Data including position, speed and acceleration of the car, and current driving mode were recorded. Inside the car, a 10.8 inch tablet which provided 9 different secondary activities was placed in the center console. During the test drive 3 TORs were issued during the test drive: (A) a 10-second TOR caused by a road narrowing provoked by a stationary car on the right lane; (B) a 10-second TOR caused by a loss of ground marking; (C) a 5-second TOR caused by a sensor failure. During the autonomous driving, participants were asked to engage in one of the secondary activities proposed by the tablet.

2 Results

To evaluate the training systems and the learning environment, objective and self-reported measures were collected anonymously and treated confidentially. A first outcome of the study is that the training allowed all the participants to respond to the TORs.

The quality of the take-over was evaluated in terms of reaction time (), the elapsed time from the TOR until the driver takes back control. A significant difference between the user manual group ( = 5.15s) and the other two systems was observed. However, no differences were observed comparing the fixed-base simulator ( = 3.17s) and the HMD ( = 3.16s). Self-reported measures were collected through set of questions at the beginning of the test, after the training and after the test drive. To evaluate the favorability of the training, the participants filled out a 10 questions survey containing questions about perceived usefulness, easiness, pleasantness and realism. In summary, according to the objective metrics measured during the test drive, the group of participants trained with the Virtual Learning Environment (fixed-base driving simulator and light VR system) were able to respond to the take-over request in less time than the group of participants trained with the user manual. Furthermore, self-reported measures showed responses in favor of the light VR training system.

3 Conclusion

The results of this research persuade us that Light Virtual Reality systems represent a valuable tool for the acquisition of driving skills in conditionally automated vehicles. The proposed training system, composed of an HMD and a game racing wheel, is a portable and cost-effective device that provides an adequate level of immersion for teaching drivers how to respond to a take-over request in a safe environment. Therefore, this system could be employed for the training of future customers of automated cars before their first ride. A direct outcome of these results is the acknowledgment of VR as key player in the definition of the set of metrics for profiling driver interaction in automated vehicles. In future work, the training will be implemented in the form of a serious game in which the level of instruction adapts to the users’ needs in order to assess the acquisition of skill during the training itself. Furthermore, test drives with the real vehicles are considered of primary importance to validate current results.

This research was supported by the French Foundation of Technological Research under grant CIFRE 2015/1392 for the doctoral work of D. Sportillo at PSA Group.


  • [1] R. Happee, C. Gold, J. Radlmayr, S. Hergeth, and K. Bengler. Take-over performance in evasive manoeuvres. Accident Analysis & Prevention, 106:211–222, 2017.
  • [2] Q. C. Ihemedu-Steinke, R. Erbach, P. Halady, G. Meixner, and M. Weber. Virtual reality driving simulator based on head-mounted displays. In Automotive User Interfaces, pp. 401–428. Springer, 2017.
  • [3] V. Melcher, S. Rauh, F. Diederichs, H. Widlroither, and W. Bauer. Take-over requests for automated driving. Procedia Manufacturing, 3:2867–2873, 2015.
  • [4] N. Merat, A. H. Jamson, F. C. Lai, M. Daly, and O. M. Carsten. Transition to manual: Driver behaviour when resuming control from a highly automated vehicle. Transportation research part F: traffic psychology and behaviour, 27:274–282, 2014.
  • [5] D. Sportillo, A. Paljic, M. Boukhris, P. Fuchs, L. Ojeda, and V. Roussarie. An immersive virtual reality system for semi-autonomous driving simulation: a comparison between realistic and 6-dof controller-based interaction. In Proceedings of the 9th International Conference on Computer and Automation Engineering, pp. 6–10. ACM, 2017.
  • [6] S. M. Taheri, K. Matsushita, and M. Sasaki. Development of a Driving Simulator with Analyzing Driver’s Characteristics Based on a Virtual Reality Head Mounted Display. Journal of Transportation Technologies, 7(03):351, 2017.
  • [7] F. Weidner, A. Hoesch, S. Poeschl, and W. Broll. Comparing VR and non-VR driving simulations: An experimental user study. In Virtual Reality (VR), 2017 IEEE, pp. 281–282. IEEE, 2017.