Discovering Closed-Loop Failures of Vision-Based Controllers via Reachability Analysis

11/04/2022
by   Kaustav Chakraborty, et al.
0

Machine learning driven image-based controllers allow robotic systems to take intelligent actions based on the visual feedback from their environment. Understanding when these controllers might lead to system safety violations is important for their integration in safety-critical applications and engineering corrective safety measures for the system. Existing methods leverage simulation-based testing (or falsification) to find the failures of vision-based controllers, i.e., the visual inputs that lead to closed-loop safety violations. However, these techniques do not scale well to the scenarios involving high-dimensional and complex visual inputs, such as RGB images. In this work, we cast the problem of finding closed-loop vision failures as a Hamilton-Jacobi (HJ) reachability problem. Our approach blends simulation-based analysis with HJ reachability methods to compute an approximation of the backward reachable tube (BRT) of the system, i.e., the set of unsafe states for the system under vision-based controllers. Utilizing the BRT, we can tractably and systematically find the system states and corresponding visual inputs that lead to closed-loop failures. These visual inputs can be subsequently analyzed to find the input characteristics that might have caused the failure. Besides its scalability to high-dimensional visual inputs, an explicit computation of BRT allows the proposed approach to capture non-trivial system failures that are difficult to expose via random simulations. We demonstrate our framework on two case studies involving an RGB image-based neural network controller for (a) autonomous indoor navigation, and (b) autonomous aircraft taxiing.

READ FULL TEXT

page 1

page 4

page 5

page 6

page 7

research
05/14/2021

Verification of Image-based Neural Network Controllers Using Generative Models

Neural networks are often used to process information from image-based s...
research
03/29/2022

NNLander-VeriF: A Neural Network Formal Verification Framework for Vision-Based Autonomous Aircraft Landing

In this paper, we consider the problem of formally verifying a Neural Ne...
research
09/25/2022

Enforcing safety for vision-based controllers via Control Barrier Functions and Neural Radiance Fields

To navigate complex environments, robots must increasingly use high-dime...
research
01/05/2021

Efficient Reachability Analysis of Closed-Loop Systems with Neural Network Controllers

Neural Networks (NNs) can provide major empirical performance improvemen...
research
03/05/2020

Validation of Image-Based Neural Network Controllers through Adaptive Stress Testing

Neural networks have become state-of-the-art for computer vision problem...
research
06/12/2021

Intelligent Vision Based Wear Forecasting on Surfaces of Machine Tool Elements

This paper addresses the ability to enable machines to automatically det...
research
09/10/2023

Certified Vision-based State Estimation for Autonomous Landing Systems using Reachability Analysis

This paper studies the problem of designing a certified vision-based sta...

Please sign up or login with your details

Forgot password? Click here to reset