The variance-penalized stochastic shortest path problem

by   Jakob Piribauer, et al.

The stochastic shortest path problem (SSPP) asks to resolve the non-deterministic choices in a Markov decision process (MDP) such that the expected accumulated weight before reaching a target state is maximized. This paper addresses the optimization of the variance-penalized expectation (VPE) of the accumulated weight, which is a variant of the SSPP in which a multiple of the variance of accumulated weights is incurred as a penalty. It is shown that the optimal VPE in MDPs with non-negative weights as well as an optimal deterministic finite-memory scheduler can be computed in exponential space. The threshold problem whether the maximal VPE exceeds a given rational is shown to be EXPTIME-hard and to lie in NEXPTIME. Furthermore, a result of interest in its own right obtained on the way is that a variance-minimal scheduler among all expectation-optimal schedulers can be computed in polynomial time.



page 1

page 3

page 5

page 7

page 9

page 11

page 15

page 17


Partial and Conditional Expectations in Markov Decision Processes with Integer Weights

The paper addresses two variants of the stochastic shortest path problem...

On Solving a Stochastic Shortest-Path Markov Decision Process as Probabilistic Inference

Previous work on planning as active inference addresses finite horizon p...

On Skolem-hardness and saturation points in Markov decision processes

The Skolem problem and the related Positivity problem for linear recurre...

Stochastic Shortest Paths and Weight-Bounded Properties in Markov Decision Processes

The paper deals with finite-state Markov decision processes (MDPs) with ...

Computational Approaches for Stochastic Shortest Path on Succinct MDPs

We consider the stochastic shortest path (SSP) problem for succinct Mark...

Risk-aware Stochastic Shortest Path

We treat the problem of risk-aware control for stochastic shortest path ...

Stochastic Shortest Path with Energy Constraints in POMDPs

We consider partially observable Markov decision processes (POMDPs) with...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.