Adaptive Stress Testing: Finding Failure Events with Reinforcement Learning

11/06/2018
by   Ritchie Lee, et al.
0

Finding the most likely path to a set of failure states is important to the analysis of safety-critical dynamic systems. While efficient solutions exist for certain classes of systems, a scalable general solution for stochastic, partially-observable, and continuous-valued systems remains challenging. Existing approaches in formal and simulation-based methods either cannot scale to large systems or are computationally inefficient. This paper presents adaptive stress testing (AST), a framework for searching a simulator for the most likely path to a failure event. We formulate the problem as a Markov decision process and use reinforcement learning to optimize it. The approach is simulation-based and does not require internal knowledge of the system. As a result, the approach is very suitable for black box testing of large systems. We present formulations for both systems where the state is fully-observable and partially-observable. In the latter case, we present a modified Monte Carlo tree search algorithm that only requires access to the pseudorandom number generator of the simulator to overcome partial observability. We also present an extension of the framework, called differential adaptive stress testing (DAST), that can be used to find failures that occur in one system but not in another. This type of differential analysis is useful in applications such as regression testing, where one is concerned with finding areas of relative weakness compared to a baseline. We demonstrate the effectiveness of the approach on an aircraft collision avoidance application, where we stress test a prototype aircraft collision avoidance system to find high-probability scenarios of near mid-air collisions.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/05/2019

Adaptive Stress Testing for Autonomous Vehicles

This paper presents a method for testing the decision making systems of ...
research
11/04/2020

Adaptive Stress Testing of Trajectory Predictions in Flight Management Systems

To find failure events and their likelihoods in flight-critical systems,...
research
07/08/2021

Adaptive Stress Testing for Adversarial Learning in a Financial Environment

We demonstrate the use of Adaptive Stress Testing to detect and address ...
research
04/01/2023

Adaptive Failure Search Using Critical States from Domain Experts

Uncovering potential failure cases is a crucial step in the validation o...
research
07/16/2019

Efficient Autonomy Validation in Simulation with Adaptive Stress Testing

During the development of autonomous systems such as driverless cars, it...
research
04/08/2020

The Adaptive Stress Testing Formulation

Validation is a key challenge in the search for safe autonomy. Simulatio...
research
10/03/2021

Stress Testing Autonomous Racing Overtake Maneuvers with RRT

High-performance autonomy often must operate at the boundaries of safety...

Please sign up or login with your details

Forgot password? Click here to reset