Automated Test Cases Prioritization for Self-driving Cars in Virtual Environments

07/20/2021
by   Christian Birchler, et al.
0

Testing with simulation environments helps to identify critical failing scenarios emerging autonomous systems such as self-driving cars (SDCs) and are safer than in-field operational tests. However, these tests are very expensive and are too many to be run frequently within limited time constraints. In this paper, we investigate test case prioritization techniques to increase the ability to detect SDC regression faults with virtual tests earlier. Our approach, called SDC-Prioritizer, prioritizes virtual tests for SDCs according to static features of the roads used within the driving scenarios. These features are collected without running the tests and do not require past execution results. SDC-Prioritizer utilizes meta-heuristics to prioritize the test cases using diversity metrics (black-box heuristics) computed on these static features. Our empirical study conducted in the SDC domain shows that SDC-Prioritizer doubles the number of safety-critical failures that virtual tests can detect at the same level of execution time compared to baselines: random and greedy-based test case orderings. Furthermore, this meta-heuristic search performs statistically better than both baselines in terms of detecting safety-critical failures. SDC-Prioritizer effectively prioritize test cases for SDCs with a large improvement in fault detection while its overhead (up to 0.34

READ FULL TEXT
research
11/08/2021

Machine Learning-based Test Selection for Simulation-based Testing of Self-driving Cars Software

Abstract Simulation platforms facilitate the development of emerging cyb...
research
07/06/2023

TEASER: Simulation-based CAN Bus Regression Testing for Self-driving Cars Software

Software systems for safety-critical systems like self-driving cars (SDC...
research
07/21/2023

Feature Map Testing for Deep Neural Networks

Due to the widespread application of deep neural networks (DNNs) in safe...
research
09/01/2018

Test Prioritization in Continuous Integration Environments

Two heuristics namely diversity-based (DBTP) and history-based test prio...
research
05/14/2019

Faster Creation of Smaller Test Suites (with SNAP)

State-of-the-art theorem provers, combined with smart sampling heuristic...
research
11/21/2022

Cost-effective Simulation-based Test Selection in Self-driving Cars Software

Simulation environments are essential for the continuous development of ...
research
02/18/2023

Practical Flaky Test Prediction using Common Code Evolution and Test History Data

Non-deterministically behaving test cases cause developers to lose trust...

Please sign up or login with your details

Forgot password? Click here to reset