Software Framework for Testing of Automated Driving Systems in a Dynamic Traffic Environment

by   Demin Nalic, et al.
TU Graz

Virtual testing of automated driving systems (ADS) has become an essential part of testing procedures for all automation levels. As ADS from automation level 3 and up are very complex, virtual testing for such systems is inevitable. The complexity of these levels lies in the modelling and calculation demand for the virtual environment which consists of roads, traffic, static and dynamic objects as well as the modelling of the car itself. For safety and performance analyses of ADS, the most important part is the modelling and consideration of road traffic participants. There is multiple traffic flow simulation software (TFSS) which are used to reproduce realistic traffic behavior and are integrated directly or over interfaces with vehicle simulation software (VSS). For these software environments, the possibility to manipulate traffic participants in a defined manner e.g. in the vicinity of the vehicle under test or implementing defined driver models for traffic vehicles is beneficial. In this paper, we present a software framework based on the external driver model interface provided by Vissim. This framework makes it possible to easily manipulate traffic participants for testing purposes of ADS.



page 1

page 2

page 3

page 4


Stress Testing Method for Scenario Based Testing of Automated Driving Systems

Classical approaches and procedures for testing of automated vehicles of...

Advanced Lane Detection Model for the Virtual Development of Highly Automated Functions

Virtual development and prototyping has already become an integral part ...

A Driver-Vehicle Model for ADS Scenario-based Testing

Scenario-based testing for automated driving systems (ADS) must be able ...

SilGAN: Generating driving maneuvers for scenario-based software-in-the-loop testing

Automotive software testing continues to rely largely upon expensive fie...

Digital Twins Are Not Monozygotic – Cross-Replicating ADAS Testing in Two Industry-Grade Automotive Simulators

The increasing levels of software- and data-intensive driving automation...

A framework for the automation of testing computer vision systems

Vision systems, i.e., systems that allow to detect and track objects in ...

OpenCDA:An Open Cooperative Driving Automation Framework Integrated with Co-Simulation

Although Cooperative Driving Automation (CDA) has attracted considerable...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

Required Metadata

Current code version

Nr. Code metadata description Please fill in this column
C1 Current code version v1
C2 Permanent link to code/repository used for this code version
C3 Code Ocean compute capsule none
C4 Legal Code License GPL-3.0 License
C5 Code versioning system used git
C6 Software code languages, tools, and services used C++, IPG CarMaker, PTV Vissim
C7 Compilation requirements, operating environments & dependencies Visual Studio
C9 Support email for questions
Table 1: Code metadata (mandatory)

1 Motivation and significance

TFSS in virtual testing of automated driving systems is already standard procedure in automotive engineering. TFSS is mostly used in combination with other software tools for vehicle testing and simulation. Therefore, there are different co-simulation platforms for testing ADS in complex traffic environments as presented in hallerbach -nalic2019framework . All of these platforms have one point in common, the vehicle under test (VUT) is developed separately in a certain vehicle software tool (VST) and the traffic is externally imported by TFSS. Using these co-simulation platforms, the performance of developed ADS in complex traffic environment can be analyzed and approved. One very common use of TFSS is the impact analysis of ADS. For that, the TFSS should offer the possibility to manipulate and edit driver models to implement ADF in certain number of the vehicles and analyze the impact of ADS on traffic; see impact1 and impact2 . The focus of this research is to test ADS within a comprehensive traffic setting using TFSS. For this research, the co-simulation between IPG CarMaker and PTV Vissim trafficFlow was used for testing ADS; see the developed co-simulation framework based on co-simulation in nalic2019framework and cosimFramework . In this co-simulation framework, the VUT, the road network and the visualization are handled by IPG CarMaker and the traffic modelled and imported from PTV Vissim. The purpose of the co-simulation framework was to detect critical scenarios for the assessment of ADS. The definition of scenarios and critical scenarios in relation with automated driving can be found in following research papers; see scenario1 -scenario2 . In general, scenarios for testing of ADS represent special or relevant maneuvers which should be tested to approve the safety performance of ADS. As all TFSS are used in the field of traffic planning and modelling for general traffic analyses, the manipulation of traffic participants for testing of ADS was not considered in the past. Due to that, the cars do not collide with each other and do unpredictable movements which could lead to a critical maneuver. PTV Vissim provides the possibility to manipulate them via an external driver model DLL (Dynamic-Link Library) interface and offers a C++ code to generate external driver models which follow a defined and developed behavior, see vissim . The C++ code of PTV Vissim is edited and adjusted in a DMF to make the implementations of external driver models easier and more adaptive. For testing purposes of ADS, the Driver Model Framework (DMF) is developed to generate more critical scenarios in the co-simulation framework from nalic2019framework and use the detected scenarios for assessment of ADS. Additionally, the developed DMF offers other usage and provides the user with the possibility to easily implement a defined vehicle behavior, automated driving functions or any other functions which are relevant for traffic modelling and analysis. By implementing the stress-testing method presented in stm in the DMF, an increase of detected and relevant scenarios for testing ADS was achieved. For further research, the DMF will be used to adjust and improve the existing testing procedure and also for traffic analyses.

2 Software description

The external driver model, DLL interface provided by PTV Vissim, contains 3 essential functions from Vissim interface : DriverModelSetValue, DriverModelGetValue and DriverModelExecuteCommand. In DMF, these functions must be available and are an essential part of it. The Vissim participants, which are controlled by the DLL, are called DLL driver models (DDM) in further description. The DriverModelExecuteCommand value is passed upon each simulation step, denoting the action to be taken, such as initialization, driver creation, driver deletion and driver movement of each traffic participant controlled by the DLL in the TFSS. In between the commands, multiple calls for DriverModelSetValue and DriverModelGetValue are made for each vehicle controlled by the DLL driver model. The function DriverModelSetValue provides the DLL with the current vehicle values which can be stored, processed and modified. In order to provide Vissim with new values which will be used for the vehicle movement, multiple calls for DriverModelGetValue for each traffic participant are necessary. For the manipulation procedure, one vehicle in Vissim is set to be the VUT. In further text the vehicle will be referred to as EGO vehicle. This vehicle has a certain ID which can be freely defined and set. For the DMF, the area of interest is the surrounding area of the EGO vehicle which consists of other traffic participants, referred to as nearby vehicles in this paper.

Each Vissim call to the set or get method passes multiple arguments, one of which always denotes the value which is going to be set or achieved. After set is executed for each value of the vehicle, multiple set calls follow with the values of the nearby vehicles. Thus, information about ego vehicle and detected nearby vehicles is obtained and can be processed and prepared if needed. The whole set and get routine is performed for each vehicle controlled by the DMF for each simulation step. To collect the data of the surrounding area of the ego vehicle in order to manipulate it, the DMF builds a wrapper around it for handling the data by the user. The DMF provides the user with the basic methods necessary to read and manipulate the traffic participants relevant for the tested ego vehicle. The user has to implement two framework methods capture and action in order to create usable DLL.

In the action method, user shall define an action which will be applied to the target vehicles. One such action can be a braking maneuver of a traffic participant which is directly in front of the ego vehicle or a cut-in of the traffic participant in front of the ego vehicle. A systematic way to manipulate traffic participants using the DMF is presented in stm . The maneuvers are executed during the predefined time window and they are not scheduled for execution until the conditions in capture are met again and new action round is started.

The obvious advantage of the framework is that user does not care about keeping all nearby vehicles, taking care of vehicle IDs or if the nearby vehicles has up to date values. In the capture method, it is guaranteed that nearby vehicles which user can observe have up to date values and those selected as target vehicles are consistently being updated during the action time window.

2.1 Software Architecture

The driver model DLL framework is written in C++ and requires user to do so. It encapsulates the inner workings and provides the user with the simple interface. The class hierarchy can be seen in Fig. 1.

Figure 1: This figure shows the simplified view of the framework class hierarhy.

The InjectorAbstract class comprises most of the DMF logic including get and set methods which are called from the Vissim. They are required to be a part of InjectorAbstract since the whole DMF logic depends on them, reading out the data, storing it, scheduling the execution of capture and action methods and passing new data to Vissim. The Injector class is a child class of the InjectorClass which implements pure virtual methods capture and action. This inheritance serves the purpose of keeping users away from the inner workings of the framework and providing them with simple means of vehicle manipulation. Through this, the user code is separated from the DMF logic and it is simple for the user to write and organize. Besides that, user is free to extend the functionality of NearbyVehicle and EgoVehicle classes which keep the data of respective vehicle categories.

2.2 Software Functionalities

As mentioned in the software description, the user has to fill the code for the capture and action methods. The methods provided by the DMF upon which user depends are:

  • getAllNearbyVehicles

  • getVehiclesDownstream

  • getVehiclesUpstream

  • getCurrentSimTime

First three methods return a vector containing nearby vehicles, either all of them, or only vehicles in front or vehicles in behind of our ego vehicle. Each vehicle is represented by the

NearbyVehicle object which contains all relevant information about the vehicle, such as relative distance and speed to the ego vehicle, acceleration, relative position, etc.. These methods are meant to be used in capture method where the user is observing the traffic and labeling relevant vehicles as target vehicles. When the user-defined criteria is fulfilled, startAction method is to be called with specified action duration time and optionally selected pause time after action end. During the action window, user is taking the control over the target vehicles and after action time expires, internal model takes over the target vehicles. Through these methods, DMF provides user with these main functionalities:

  • Providing the user with the up to date list of nearby vehicles

  • Letting the user to select target vehicles and manipulate them

  • Keeping the list of target vehicles updated during the action

  • Scheduling the capture and action methods in time

  • Taking and releasing the control of the vehicles from and to the user

Another important feature of the DMF is the two separate modes of operation. In both of them previously defined functionalities are present, but they require different approach and implementation for different modes of operation. The DMF can operate both in Vissim only and in co-simulation between CarMaker and Vissim.

3 Impact

  • The presented DMF provides the possibility to manipulate traffic participants in the TFSS of Vissim. By implementing automated driving functions using the DLL an impact analysis of these functions on traffic can be analyzed and evaluated.

  • The possibility to manipulate a certain traffic participant in the surrounding area of a vehicle under test can be used to provoke critical scenarios. Assuming that the vehicle under test is equipped with ADF, the provoked critical scenarios can be used to detect, assess and verify the ADS implemented in the vehicle under test.

  • The increase of relevant scenarios for testing automated can be achieved; see stm .

  • Using the DMF additional test procedures for testing stability or testing platooning systems can and will be defined and implemented for future works.

4 Conclusions

With the presented software framework, the user has the possibility to easily implement and adjust driver models for traffic participants using the traffic flow simulation software Vissim. That makes it possible to analyze traffic behavior of implemented models or to manipulate traffic participants for testing procedures of automated driving systems. Using the co-simulation between Vissim and IPG CarMaker with the adjustment of the presented framework, it was possible to upgrade the benefit of co-simulation for testing of automated driving systems.

5 Conflict of Interest

No conflict of interest exists: We wish to confirm that there are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome.