SOS: Safe, Optimal and Small Strategies for Hybrid Markov Decision Processes

06/25/2019
by   Pranav Ashok, et al.
0

For hybrid Markov decision processes, UPPAAL Stratego can compute strategies that are safe for a given safety property and (in the limit) optimal for a given cost function. Unfortunately, these strategies cannot be exported easily since they are computed as a very long list. In this paper, we demonstrate methods to learn compact representations of the strategies in the form of decision trees. These decision trees are much smaller, more understandable, and can easily be exported as code that can be loaded into embedded systems. Despite the size compression and actual differences to the original strategy, we provide guarantees on both safety and optimality of the decision-tree strategy. On the top, we show how to obtain yet smaller representations, which are still guaranteed safe, but achieve a desired trade-off between size and optimality.

READ FULL TEXT

page 13

page 20

research
01/30/2023

Optimal Decision Tree Policies for Markov Decision Processes

Interpretability of reinforcement learning policies is essential for man...
research
04/01/2019

Efficient and Safe Exploration in Deterministic Markov Decision Processes with Unknown Transition Models

We propose a safe exploration algorithm for deterministic Markov Decisio...
research
06/19/2019

Strategy Representation by Decision Trees with Linear Classifiers

Graph games and Markov decision processes (MDPs) are standard models in ...
research
11/22/2021

Strategies for the Iterated Prisoner's Dilemma

We explore some strategies which tend to perform well in the IPD. We sta...
research
02/02/2018

Strategy Representation by Decision Trees in Reactive Synthesis

Graph games played by two players over finite-state graphs are central i...
research
10/22/2020

Query strategies for priced information, revisited

We consider the problem of designing query strategies for priced informa...
research
06/22/2011

The Communicative Multiagent Team Decision Problem: Analyzing Teamwork Theories and Models

Despite the significant progress in multiagent teamwork, existing resear...

Please sign up or login with your details

Forgot password? Click here to reset