Reachable Set Estimation for Neural Network Control Systems: A Simulation-Guided Approach

04/26/2020
by   Weiming Xiang, et al.
0

The vulnerability of artificial intelligence (AI) and machine learning (ML) against adversarial disturbances and attacks significantly restricts their applicability in safety-critical systems including cyber-physical systems (CPS) equipped with neural network components at various stages of sensing and control. This paper addresses the reachable set estimation and safety verification problems for dynamical systems embedded with neural network components serving as feedback controllers. The closed-loop system can be abstracted in the form of a continuous-time sampled-data system under the control of a neural network controller. First, a novel reachable set computation method in adaptation to simulations generated out of neural networks is developed. The reachability analysis of a class of feedforward neural networks called multilayer perceptrons (MLP) with general activation functions is performed in the framework of interval arithmetic. Then, in combination with reachability methods developed for various dynamical system classes modeled by ordinary differential equations, a recursive algorithm is developed for over-approximating the reachable set of the closed-loop system. The safety verification for neural network control systems can be performed by examining the emptiness of the intersection between the over-approximation of reachable sets and unsafe sets. The effectiveness of the proposed approach has been validated with evaluations on a robotic arm model and an adaptive cruise control system.

READ FULL TEXT
research
04/12/2020

NNV: The Neural Network Verification Tool for Deep Neural Networks and Learning-Enabled Cyber-Physical Systems

This paper presents the Neural Network Verification (NNV) software tool,...
research
12/14/2018

Specification-Guided Safety Verification for Feedforward Neural Networks

This paper presents a specification-guided safety verification method fo...
research
12/14/2022

Automated Reachability Analysis of Neural Network-Controlled Systems via Adaptive Polytopes

Over-approximating the reachable sets of dynamical systems is a fundamen...
research
07/13/2022

Reachability Analysis of a General Class of Neural Ordinary Differential Equations

Continuous deep learning models, referred to as Neural Ordinary Differen...
research
12/16/2021

Verification of Neural-Network Control Systems by Integrating Taylor Models and Zonotopes

We study the verification problem for closed-loop dynamical systems with...
research
10/03/2018

Verification for Machine Learning, Autonomy, and Neural Networks Survey

This survey presents an overview of verification techniques for autonomo...
research
01/17/2023

Safety Verification of Neural Network Control Systems Using Guaranteed Neural Network Model Reduction

This paper aims to enhance the computational efficiency of safety verifi...

Please sign up or login with your details

Forgot password? Click here to reset