DeepAI AI Chat
Log In Sign Up

Confidence Composition for Monitors of Verification Assumptions

by   Ivan Ruchkin, et al.

Closed-loop verification of cyber-physical systems with neural network controllers offers strong safety guarantees under certain assumptions. It is, however, difficult to determine whether these guarantees apply at run time because verification assumptions may be violated. To predict safety violations in a verified system, we propose a three-step framework for monitoring the confidence in verification assumptions. First, we represent the sufficient condition for verified safety with a propositional logical formula over assumptions. Second, we build calibrated confidence monitors that evaluate the probability that each assumption holds. Third, we obtain the confidence in the verification guarantees by composing the assumption monitors using a composition function suitable for the logical formula. Our framework provides theoretical bounds on the calibration and conservatism of compositional monitors. In two case studies, we demonstrate that the composed monitors improve over their constituents and successfully predict safety violations.


page 1

page 2

page 3

page 4


Overview of Logical Foundations of Cyber-Physical Systems

Cyber-physical systems (CPSs) are important whenever computer technology...

Formal Verification of Stochastic Systems with ReLU Neural Network Controllers

In this work, we address the problem of formal safety verification for s...

Formal Verification of End-to-End Learning in Cyber-Physical Systems: Progress and Challenges

Autonomous systems – such as self-driving cars, autonomous drones, and a...

Verified Runtime Validation for Partially Observable Hybrid Systems

Formal verification provides strong safety guarantees about models of cy...

Compositional Cyber-Physical Systems Theory

This dissertation builds a compositional cyber-physical systems theory t...

Stress Testing of Design Assumptions in Cyper-Physical Systems: A Control Theory-Based Approach

Cyber-Physical Systems (CPS) are most of the time safety-critical and ex...

A Verification Framework for Certifying Learning-Based Safety-Critical Aviation Systems

We present a safety verification framework for design-time and run-time ...