Automated Test Cases Prioritization for Self-driving Cars in Virtual Environments

by   Christian Birchler, et al.

Testing with simulation environments helps to identify critical failing scenarios emerging autonomous systems such as self-driving cars (SDCs) and are safer than in-field operational tests. However, these tests are very expensive and are too many to be run frequently within limited time constraints. In this paper, we investigate test case prioritization techniques to increase the ability to detect SDC regression faults with virtual tests earlier. Our approach, called SDC-Prioritizer, prioritizes virtual tests for SDCs according to static features of the roads used within the driving scenarios. These features are collected without running the tests and do not require past execution results. SDC-Prioritizer utilizes meta-heuristics to prioritize the test cases using diversity metrics (black-box heuristics) computed on these static features. Our empirical study conducted in the SDC domain shows that SDC-Prioritizer doubles the number of safety-critical failures that virtual tests can detect at the same level of execution time compared to baselines: random and greedy-based test case orderings. Furthermore, this meta-heuristic search performs statistically better than both baselines in terms of detecting safety-critical failures. SDC-Prioritizer effectively prioritize test cases for SDCs with a large improvement in fault detection while its overhead (up to 0.34


Machine Learning-based Test Selection for Simulation-based Testing of Self-driving Cars Software

Abstract Simulation platforms facilitate the development of emerging cyb...

scenoRITA: Generating Less-Redundant, Safety-Critical and Motion Sickness-Inducing Scenarios for Autonomous Vehicles

There is tremendous global enthusiasm for research, development, and dep...

Automatically Generating Test Cases for Safety-Critical Software via Symbolic Execution

Automated test generation based on symbolic execution can be beneficial ...

Test Prioritization in Continuous Integration Environments

Two heuristics namely diversity-based (DBTP) and history-based test prio...

Faster Creation of Smaller Test Suites (with SNAP)

State-of-the-art theorem provers, combined with smart sampling heuristic...

An Empirical Testing of Autonomous Vehicle Simulator System for Urban Driving

Safety is one of the main challenges that prohibit autonomous vehicles (...

Toward Speeding up Mutation Analysis by Memoizing Expensive Methods

Mutation analysis has many applications, such as assessing the quality o...