
Maximum Expected Hitting Cost of a Markov Decision Process and Informativeness of Rewards
We propose a new complexity measure for Markov decision processes (MDP),...
read it

Markov Rewards Processes with Impulse Rewards and Absorbing States
We study the expected accumulated reward for a discretetime Markov rewa...
read it

Graph Planning with Expected Finite Horizon
Graph planning gives rise to fundamental algorithmic questions such as s...
read it

Computing the Expected Execution Time of Probabilistic Workflow Nets
FreeChoice Workflow Petri nets, also known as Workflow Graphs, are a po...
read it

The Markovian Price of Information
Suppose there are n Markov chains and we need to pay a perstep price to...
read it

Partial and Conditional Expectations in Markov Decision Processes with Integer Weights
The paper addresses two variants of the stochastic shortest path problem...
read it

Decisiveness of Stochastic Systems and its Application to Hybrid Models (Full Version)
In [ABM07], Abdulla et al. introduced the concept of decisiveness, an in...
read it
Stochastic Processes with Expected Stopping Time
Markov chains are the de facto finitestate model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating nondeterministic behaviors. Given an MDP and rewards on states, a classical optimization criterion is the maximal expected total reward where the MDP stops after T steps, which can be computed by a simple dynamic programming algorithm. We consider a natural generalization of the problem where the stopping times can be chosen according to a probability distribution, such that the expected stopping time is T, to optimize the expected total reward. Quite surprisingly we establish interreducibility of the expected stoppingtime problem for Markov chains with the Positivity problem (which is related to the wellknown Skolem problem), for which establishing either decidability or undecidability would be a major breakthrough. Given the hardness of the exact problem, we consider the approximate version of the problem: we show that it can be solved in exponential time for Markov chains and in exponential space for MDPs.
READ FULL TEXT
Comments
There are no comments yet.