Leveraging Classification Metrics for Quantitative System-Level Analysis with Temporal Logic Specifications

by   Apurva Badithela, et al.

In many autonomy applications, performance of perception algorithms is important for effective planning and control. In this paper, we introduce a framework for computing the probability of satisfaction of formal system specifications given a confusion matrix, a statistical average performance measure for multi-class classification. We define the probability of satisfaction of a linear temporal logic formula given a specific initial state of the agent and true state of the environment. Then, we present an algorithm to construct a Markov chain that represents the system behavior under the composition of the perception and control components such that the probability of the temporal logic formula computed over the Markov chain is consistent with the probability that the temporal logic formula is satisfied by our system. We illustrate this approach on a simple example of a car with pedestrian on the sidewalk environment, and compute the probability of satisfaction of safety requirements for varying parameters of the vehicle. We also illustrate how satisfaction probability changes with varied precision and recall derived from the confusion matrix. Based on our results, we identify several opportunities for future work in developing quantitative system-level analysis that incorporates perception models.



There are no comments yet.


page 1


Secure Control in Partially Observable Environments to Satisfy LTL Specifications

This paper studies the synthesis of control policies for an agent that h...

Elliptical Slice Sampling for Probabilistic Verification of Stochastic Systems with Signal Temporal Logic Specifications

Autonomous robots typically incorporate complex sensors in their decisio...

Maximum Realizability for Linear Temporal Logic Specifications

Automatic synthesis from linear temporal logic (LTL) specifications is w...

Secure Control under Partial Observability with Temporal Logic Constraints

This paper studies the synthesis of control policies for an agent that h...

Distributed Control using Reinforcement Learning with Temporal-Logic-Based Reward Shaping

We present a computational framework for synthesis of distributed contro...

Signal Temporal Logic Task Decomposition via Convex Optimization

In this paper we focus on the problem of decomposing a global Signal Tem...

Understanding the Dynamics of Information Flow During Disaster Response Using Absorbing Markov Chains

This paper aims to derive a quantitative model to evaluate the impact of...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.