Geometric fluid approximation for general continuous-time Markov chains

01/31/2019
by   Michalis Michaelides, et al.
6

Fluid approximations have seen great success in approximating the macro-scale behaviour of Markov systems with a large number of discrete states. However, these methods rely on the continuous-time Markov chain (CTMC) having a particular population structure which suggests a natural continuous state-space endowed with a dynamics for the approximating process. We construct here a general method based on spectral analysis of the transition matrix of the CTMC, without the need for a population structure. Specifically, we use the popular manifold learning method of diffusion maps to analyse the transition matrix as the operator of a hidden continuous process. An embedding of states in a continuous space is recovered, and the space is endowed with a drift vector field inferred via Gaussian process regression. In this manner, we construct an ODE whose solution approximates the evolution of the CTMC mean, mapped onto the continuous space (known as the fluid limit).

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/04/2018

An Imprecise Probabilistic Estimator for the Transition Rate Matrix of a Continuous-Time Markov Chain

We consider the problem of estimating the transition rate matrix of a co...
research
03/19/2020

EM algorithm for stochastic hybrid systems

A stochastic hybrid system, also known as a switching diffusion, is a co...
research
06/24/2019

Bayesian Nonparametric Clustering of Continuous-Time Hidden Markov Models for Health Trajectories

We develop clustering procedures for healthcare trajectories based on a ...
research
04/23/2018

Central Limit Model Checking

We consider probabilistic model checking for continuous-time Markov chai...
research
06/03/2016

Property-driven State-Space Coarsening for Continuous Time Markov Chains

Dynamical systems with large state-spaces are often expensive to thoroug...
research
04/01/2018

Toroidal diffusions and protein structure evolution

This chapter shows how toroidal diffusions are convenient methodological...
research
11/10/2017

Model Checking Markov Population Models by Stochastic Approximations

Many complex systems can be described by population models, in which a p...

Please sign up or login with your details

Forgot password? Click here to reset