DeepAI AI Chat
Log In Sign Up

Scenario-Based Verification of Uncertain MDPs

by   Murat Cubuktepe, et al.

We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of random variables. The probability distributions for these random parameters are unknown. The problem is to compute the probability to satisfy a temporal logic specification within any MDP that corresponds to a sample from these unknown distributions. In general, this problem is undecidable, and we resort to techniques from so-called scenario optimization. Based on a finite number of samples of the uncertain parameters, each of which induces an MDP, the proposed method estimates the probability of satisfying the specification by solving a finite-dimensional convex optimization problem. The number of samples required to obtain a high confidence on this estimate is independent from the number of states and the number of random parameters. Experiments on a large set of benchmarks show that a few thousand samples suffice to obtain high-quality confidence bounds with a high probability.


page 1

page 2

page 3

page 4


Scenario-Based Verification of Uncertain Parametric MDPs

We consider parametric Markov decision processes (pMDPs) that are augmen...

Convex Optimization for Parameter Synthesis in MDPs

Probabilistic model checking aims to prove whether a Markov decision pro...

Robust Finite-State Controllers for Uncertain POMDPs

Uncertain partially observable Markov decision processes (uPOMDPs) allow...

Robust Combination of Local Controllers

Planning problems are hard, motion planning, for example, isPSPACE-hard....

Sampling-Based Verification of CTMCs with Uncertain Rates

We employ uncertain parametric CTMCs with parametric transition rates an...

Computation of Reliability Statistics for Finite Samples of Success-Failure Experiments

Computational method for statistical measures of reliability, confidence...

Learning-Based Mean-Payoff Optimization in an Unknown MDP under Omega-Regular Constraints

We formalize the problem of maximizing the mean-payoff value with high p...