Qualitative Controller Synthesis for Consumption Markov Decision Processes

05/14/2020
by   František Blahoudek, et al.
0

Consumption Markov Decision Processes (CMDPs) are probabilistic decision-making models of resource-constrained systems. In a CMDP, the controller possesses a certain amount of a critical resource, such as electric power. Each action of the controller can consume some amount of the resource. Resource replenishment is only possible in special reload states, in which the resource level can be reloaded up to the full capacity of the system. The task of the controller is to prevent resource exhaustion, i.e. ensure that the available amount of the resource stays non-negative, while ensuring an additional linear-time property. We study the complexity of strategy synthesis in consumption MDPs with almost-sure Büchi objectives. We show that the problem can be solved in polynomial time. We implement our algorithm and show that it can efficiently solve CMDPs modelling real-world scenarios.

READ FULL TEXT

page 1

page 2

page 3

page 4

05/05/2021

Efficient Strategy Synthesis for MDPs with Resource Constraints

We consider qualitative strategy synthesis for the formalism called cons...
05/04/2021

Polynomial-Time Algorithms for Multi-Agent Minimal-Capacity Planning

We study the problem of minimizing the resource capacity of autonomous a...
12/20/2017

Temporal logic control of general Markov decision processes by approximate policy refinement

The formal verification and controller synthesis for Markov decision pro...
09/10/2018

Multi-weighted Markov Decision Processes with Reachability Objectives

In this paper, we are interested in the synthesis of schedulers in doubl...
07/24/2019

An Overview for Markov Decision Processes in Queues and Networks

Markov decision processes (MDPs) in queues and networks have been an int...
02/17/2021

Self-Triggered Markov Decision Processes

In this paper, we study Markov Decision Processes (MDPs) with self-trigg...
05/10/2021

Multi-Objective Controller Synthesis with Uncertain Human Preferences

Multi-objective controller synthesis concerns the problem of computing a...