SOS: Safe, Optimal and Small Strategies for Hybrid Markov Decision Processes

by   Pranav Ashok, et al.

For hybrid Markov decision processes, UPPAAL Stratego can compute strategies that are safe for a given safety property and (in the limit) optimal for a given cost function. Unfortunately, these strategies cannot be exported easily since they are computed as a very long list. In this paper, we demonstrate methods to learn compact representations of the strategies in the form of decision trees. These decision trees are much smaller, more understandable, and can easily be exported as code that can be loaded into embedded systems. Despite the size compression and actual differences to the original strategy, we provide guarantees on both safety and optimality of the decision-tree strategy. On the top, we show how to obtain yet smaller representations, which are still guaranteed safe, but achieve a desired trade-off between size and optimality.


page 13

page 20


Efficient and Safe Exploration in Deterministic Markov Decision Processes with Unknown Transition Models

We propose a safe exploration algorithm for deterministic Markov Decisio...

Strategy Representation by Decision Trees with Linear Classifiers

Graph games and Markov decision processes (MDPs) are standard models in ...

Strategies for the Iterated Prisoner's Dilemma

We explore some strategies which tend to perform well in the IPD. We sta...

Strategy Representation by Decision Trees in Reactive Synthesis

Graph games played by two players over finite-state graphs are central i...

Query strategies for priced information, revisited

We consider the problem of designing query strategies for priced informa...

Comparing discounted and average-cost Markov Decision Processes: a statistical significance perspective

Optimal Markov Decision Process policies for problems with finite state ...

The Communicative Multiagent Team Decision Problem: Analyzing Teamwork Theories and Models

Despite the significant progress in multiagent teamwork, existing resear...