The variance-penalized stochastic shortest path problem

04/21/2022
by   Jakob Piribauer, et al.
0

The stochastic shortest path problem (SSPP) asks to resolve the non-deterministic choices in a Markov decision process (MDP) such that the expected accumulated weight before reaching a target state is maximized. This paper addresses the optimization of the variance-penalized expectation (VPE) of the accumulated weight, which is a variant of the SSPP in which a multiple of the variance of accumulated weights is incurred as a penalty. It is shown that the optimal VPE in MDPs with non-negative weights as well as an optimal deterministic finite-memory scheduler can be computed in exponential space. The threshold problem whether the maximal VPE exceeds a given rational is shown to be EXPTIME-hard and to lie in NEXPTIME. Furthermore, a result of interest in its own right obtained on the way is that a variance-minimal scheduler among all expectation-optimal schedulers can be computed in polynomial time.

READ FULL TEXT

Authors

page 1

page 3

page 5

page 7

page 9

page 11

page 15

page 17

02/12/2019

Partial and Conditional Expectations in Markov Decision Processes with Integer Weights

The paper addresses two variants of the stochastic shortest path problem...
09/13/2021

On Solving a Stochastic Shortest-Path Markov Decision Process as Probabilistic Inference

Previous work on planning as active inference addresses finite horizon p...
04/23/2020

On Skolem-hardness and saturation points in Markov decision processes

The Skolem problem and the related Positivity problem for linear recurre...
04/30/2018

Stochastic Shortest Paths and Weight-Bounded Properties in Markov Decision Processes

The paper deals with finite-state Markov decision processes (MDPs) with ...
04/24/2018

Computational Approaches for Stochastic Shortest Path on Succinct MDPs

We consider the stochastic shortest path (SSP) problem for succinct Mark...
03/03/2022

Risk-aware Stochastic Shortest Path

We treat the problem of risk-aware control for stochastic shortest path ...
02/24/2016

Stochastic Shortest Path with Energy Constraints in POMDPs

We consider partially observable Markov decision processes (POMDPs) with...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.