scenoRITA: Generating Less-Redundant, Safety-Critical and Motion Sickness-Inducing Scenarios for Autonomous Vehicles

by   Sumaya Almanee, et al.

There is tremendous global enthusiasm for research, development, and deployment of autonomous vehicles (AVs), e.g., self-driving taxis and trucks from Waymo and Baidu. The current practice for testing AVs uses virtual tests-where AVs are tested in software simulations-since they offer a more efficient and safer alternative compared to field operational tests. Specifically, search-based approaches are used to find particularly critical situations. These approaches provide an opportunity to automatically generate tests; however, systematically creating valid and effective tests for AV software remains a major challenge. To address this challenge, we introduce scenoRITA, a test generation approach for AVs that uses evolutionary algorithms with (1) a novel gene representation that allows obstacles to be fully mutable, hence, resulting in more reported violations, (2) 5 test oracles to determine both safety and motion sickness-inducing violations, and (3) a novel technique to identify and eliminate duplicate tests. Our extensive evaluation shows that scenoRITA can produce effective driving scenarios that expose an ego car to safety critical situations. scenoRITA generated tests that resulted in a total of 1,026 unique violations, increasing the number of reported violations by 23.47 partially-mutable test generation, respectively.


"Curse of rarity" for autonomous vehicles

Achieving the human-level safety performance for autonomous vehicles (AV...

SafeBench: A Benchmarking Platform for Safety Evaluation of Autonomous Vehicles

As shown by recent studies, machine intelligence-enabled systems are vul...

Automated Test Cases Prioritization for Self-driving Cars in Virtual Environments

Testing with simulation environments helps to identify critical failing ...

Corner Case Generation and Analysis for Safety Assessment of Autonomous Vehicles

Testing and evaluation is a crucial step in the development and deployme...

An Agency-Directed Approach to Test Generation for Simulation-based Autonomous Vehicle Verification

Simulation-based verification is beneficial for assessing otherwise dang...

Neural Network Guided Evolutionary Fuzzing for Finding Traffic Violations of Autonomous Vehicles

Self-driving cars and trucks, autonomous vehicles (AVs), should not be a...

Assessing Safety-Critical Systems from Operational Testing: A Study on Autonomous Vehicles

Context: Demonstrating high reliability and safety for safety-critical s...