SOS: Safe, Optimal and Small Strategies for Hybrid Markov Decision Processes

06/25/2019
by   Pranav Ashok, et al.
0

For hybrid Markov decision processes, UPPAAL Stratego can compute strategies that are safe for a given safety property and (in the limit) optimal for a given cost function. Unfortunately, these strategies cannot be exported easily since they are computed as a very long list. In this paper, we demonstrate methods to learn compact representations of the strategies in the form of decision trees. These decision trees are much smaller, more understandable, and can easily be exported as code that can be loaded into embedded systems. Despite the size compression and actual differences to the original strategy, we provide guarantees on both safety and optimality of the decision-tree strategy. On the top, we show how to obtain yet smaller representations, which are still guaranteed safe, but achieve a desired trade-off between size and optimality.

READ FULL TEXT

page 13

page 20

04/01/2019

Efficient and Safe Exploration in Deterministic Markov Decision Processes with Unknown Transition Models

We propose a safe exploration algorithm for deterministic Markov Decisio...
06/19/2019

Strategy Representation by Decision Trees with Linear Classifiers

Graph games and Markov decision processes (MDPs) are standard models in ...
11/22/2021

Strategies for the Iterated Prisoner's Dilemma

We explore some strategies which tend to perform well in the IPD. We sta...
02/02/2018

Strategy Representation by Decision Trees in Reactive Synthesis

Graph games played by two players over finite-state graphs are central i...
10/22/2020

Query strategies for priced information, revisited

We consider the problem of designing query strategies for priced informa...
12/01/2021

Comparing discounted and average-cost Markov Decision Processes: a statistical significance perspective

Optimal Markov Decision Process policies for problems with finite state ...
06/22/2011

The Communicative Multiagent Team Decision Problem: Analyzing Teamwork Theories and Models

Despite the significant progress in multiagent teamwork, existing resear...