DeepAI AI Chat
Log In Sign Up

Scenario-Based Verification of Uncertain MDPs

12/24/2019
by   Murat Cubuktepe, et al.
0

We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of random variables. The probability distributions for these random parameters are unknown. The problem is to compute the probability to satisfy a temporal logic specification within any MDP that corresponds to a sample from these unknown distributions. In general, this problem is undecidable, and we resort to techniques from so-called scenario optimization. Based on a finite number of samples of the uncertain parameters, each of which induces an MDP, the proposed method estimates the probability of satisfying the specification by solving a finite-dimensional convex optimization problem. The number of samples required to obtain a high confidence on this estimate is independent from the number of states and the number of random parameters. Experiments on a large set of benchmarks show that a few thousand samples suffice to obtain high-quality confidence bounds with a high probability.

READ FULL TEXT

page 1

page 2

page 3

page 4

12/24/2021

Scenario-Based Verification of Uncertain Parametric MDPs

We consider parametric Markov decision processes (pMDPs) that are augmen...
06/30/2021

Convex Optimization for Parameter Synthesis in MDPs

Probabilistic model checking aims to prove whether a Markov decision pro...
09/24/2020

Robust Finite-State Controllers for Uncertain POMDPs

Uncertain partially observable Markov decision processes (uPOMDPs) allow...
01/10/2013

Robust Combination of Local Controllers

Planning problems are hard, motion planning, for example, isPSPACE-hard....
05/17/2022

Sampling-Based Verification of CTMCs with Uncertain Rates

We employ uncertain parametric CTMCs with parametric transition rates an...
05/26/2023

Computation of Reliability Statistics for Finite Samples of Success-Failure Experiments

Computational method for statistical measures of reliability, confidence...
04/24/2018

Learning-Based Mean-Payoff Optimization in an Unknown MDP under Omega-Regular Constraints

We formalize the problem of maximizing the mean-payoff value with high p...