PAC Statistical Model Checking for Markov Decision Processes and Stochastic Games
share
Statistical model checking (SMC) is a technique for analysis of probabilistic systems that may be (partially) unknown. We present an SMC algorithm for (unbounded) reachability yielding probably approximately correct (PAC) guarantees on the results. On the one hand, it is the first such algorithm for stochastic games. On the other hand, it is the first practical algorithm with such guarantees even for Markov decision processes. Compared to previous approaches where PAC guarantees require running times longer than the age of universe even for systems with a handful of states, our algorithm often yields reasonably precise results within minutes. We consider both the setting (i) with no knowledge of the transition function and (ii) with knowledge of the topology of the underlying graph.
READ FULL TEXT