Computing bounds for imprecise continuous-time Markov chains using normal cones

by   Damjan Škulj, et al.

The theory of imprecise Markov chains has achieved significant progress in recent years. Its applicability, however, is still very much limited, due in large part to the lack of efficient computational methods for calculating higher-dimensional models. The high computational complexity shows itself especially in the calculation of the imprecise version of the Kolmogorov backward equation. The equation is represented at every point of an interval in the form of a minimization problem, solvable merely with linear programming techniques. Consequently, finding an exact solution on an entire interval is infeasible, whence approximation approaches have been developed. To achieve sufficient accuracy, in general, the linear programming optimization methods need to be used in a large number of time points. The principal goal of this paper is to provide a new, more efficient approach for solving the imprecise Kolmogorov backward equation. It is based on the Lipschitz continuity of the solutions of the equation with respect to time, causing the linear programming problems appearing in proximate points of the time interval to have similar optimal solutions. This property is exploited by utilizing the theory of normal cones of convex sets. The present article is primarily devoted to providing the theoretical basis for the novel technique, yet, the initial testing shows that in most cases it decisively outperforms the existing methods.



There are no comments yet.


page 1

page 2

page 3

page 4


Duality Gap in Interval Linear Programming

This paper deals with the problem of linear programming with inexact dat...

Solving Factored MDPs with Hybrid State and Action Variables

Efficient representations and solutions for large decision problems with...

Dynamic Programming and Linear Programming for Odds Problem

This paper discusses the odds problem, proposed by Bruss in 2000, and it...

Optimal Tuning of Two-Dimensional Keyboards

We give a new analysis of a tuning problem in music theory, pertaining s...

Bounding Mean First Passage Times in Population Continuous-Time Markov Chains

We consider the problem of bounding mean first passage times for a class...

Convex Q-Learning, Part 1: Deterministic Optimal Control

It is well known that the extension of Watkins' algorithm to general fun...

On preconditioning and solving an extended class of interval parametric linear systems

We deal with interval parametric systems of linear equations and the goa...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.