Self-Triggered Markov Decision Processes

02/17/2021
by   Yunhan Huang, et al.
0

In this paper, we study Markov Decision Processes (MDPs) with self-triggered strategies, where the idea of self-triggered control is extended to more generic MDP models. This extension broadens the application of self-triggering policies to a broader range of systems. We study the co-design problems of the control policy and the triggering policy to optimize two pre-specified cost criteria. The first cost criterion is introduced by incorporating a pre-specified update penalty into the traditional MDP cost criteria to reduce the use of communication resources. Under this criteria, a novel dynamic programming (DP) equation called DP equation with optimized lookahead to proposed to solve for the self-triggering policy under this criteria. The second self-triggering policy is to maximize the triggering time while still guaranteeing a pre-specified level of sub-optimality. Theoretical underpinnings are established for the computation and implementation of both policies. Through a gridworld numerical example, we illustrate the two policies' effectiveness in reducing sources consumption and demonstrate the trade-offs between resource consumption and system performance.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/02/2021

Stability-Constrained Markov Decision Processes Using MPC

In this paper, we consider solving discounted Markov Decision Processes ...
research
09/30/2022

Prioritizing emergency evacuations under compounding levels of uncertainty

Well-executed emergency evacuations can save lives and reduce suffering....
research
09/20/2018

Logically-Constrained Neural Fitted Q-Iteration

This paper proposes a method for efficient training of the Q-function fo...
research
10/17/2022

Risk-Sensitive Markov Decision Processes with Long-Run CVaR Criterion

CVaR (Conditional Value at Risk) is a risk metric widely used in finance...
research
12/08/2020

Minimax Regret Optimisation for Robust Planning in Uncertain Markov Decision Processes

The parameters for a Markov Decision Process (MDP) often cannot be speci...
research
07/12/2022

Compactly Restrictable Metric Policy Optimization Problems

We study policy optimization problems for deterministic Markov decision ...
research
01/09/2020

Self-guided Approximate Linear Programs

Approximate linear programs (ALPs) are well-known models based on value ...

Please sign up or login with your details

Forgot password? Click here to reset