PAC Statistical Model Checking for Markov Decision Processes and Stochastic Games

05/10/2019
by   Pranav Ashok, et al.
0

Statistical model checking (SMC) is a technique for analysis of probabilistic systems that may be (partially) unknown. We present an SMC algorithm for (unbounded) reachability yielding probably approximately correct (PAC) guarantees on the results. On the one hand, it is the first such algorithm for stochastic games. On the other hand, it is the first practical algorithm with such guarantees even for Markov decision processes. Compared to previous approaches where PAC guarantees require running times longer than the age of universe even for systems with a handful of states, our algorithm often yields reasonably precise results within minutes. We consider both the setting (i) with no knowledge of the transition function and (ii) with knowledge of the topology of the underlying graph.

READ FULL TEXT
research
06/03/2022

PAC Statistical Model Checking of Mean Payoff in Discrete- and Continuous-Time MDP

Markov decision processes (MDP) and continuous-time MDP (CTMDP) are the ...
research
05/26/2021

Runtime Monitoring for Markov Decision Processes

We investigate the problem of monitoring partially observable systems wi...
research
08/24/2020

Taming denumerable Markov decision processes with decisiveness

Decisiveness has proven to be an elegant concept for denumerable Markov ...
research
07/04/2021

Linear-Time Model Checking Branching Processes

(Multi-type) branching processes are a natural and well-studied model fo...
research
09/05/2020

PAC Reinforcement Learning Algorithm for General-Sum Markov Games

This paper presents a theoretical framework for probably approximately c...
research
04/17/2023

Scenario Approach for Parametric Markov Models

In this paper, we propose an approximating framework for analyzing param...
research
07/17/2023

Compositional Probabilistic Model Checking with String Diagrams of MDPs

We present a compositional model checking algorithm for Markov decision ...

Please sign up or login with your details

Forgot password? Click here to reset