On Convergence of General Truncation-Augmentation Schemes for Approximating Stationary Distributions of Markov Chains

03/29/2022
by   Alex Infanger, et al.
0

In the analysis of Markov chains and processes, it is sometimes convenient to replace an unbounded state space with a "truncated" bounded state space. When such a replacement is made, one often wants to know whether the equilibrium behavior of the truncated chain or process is close to that of the untruncated system. For example, such questions arise naturally when considering numerical methods for computing stationary distributions on unbounded state space. In this paper, we study general truncation-augmentation schemes, in which the substochastic truncated "northwest corner" of the transition matrix or kernel is stochasticized (or augmented) arbitrarily. In the presence of a Lyapunov condition involving a coercive function, we show that such schemes are generally convergent in countable state space, provided that the truncation is chosen as a sublevel set of the Lyapunov function. For stochastically monotone Markov chains on ℤ_+, we prove that we can always choose the truncation sets to be of the form {0,1,...,n}. We then provide sufficient conditions for weakly continuous Markov chains under which general truncation-augmentation schemes converge weakly in continuous state space. Finally, we briefly discuss the extension of the theory to continuous time Markov jump processes.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/23/2022

On Convergence of a Truncation Scheme for Approximating Stationary Distributions of Continuous State Space Markov Chains and Processes

In the analysis of Markov chains and processes, it is sometimes convenie...
research
09/02/2018

Stable approximation schemes for optimal filters

We explore a general truncation scheme for the approximation of (possibl...
research
11/11/2019

Markov chains in random environment with applications in queueing theory and machine learning

We prove the existence of limiting distributions for a large class of Ma...
research
08/30/2022

A New Truncation Algorithm for Markov Chain Equilibrium Distributions with Computable Error Bounds

This paper introduces a new algorithm for numerically computing equilibr...
research
06/13/2019

Hypotheses testing and posterior concentration rates for semi-Markov processes

In this paper, we adopt a nonparametric Bayesian approach and investigat...
research
08/02/2016

Rate Reduction for State-labelled Markov Chains with Upper Time-bounded CSL Requirements

This paper presents algorithms for identifying and reducing a dedicated ...
research
12/21/2021

Differentiated uniformization: A new method for inferring Markov chains on combinatorial state spaces including stochastic epidemic models

Motivation: We consider continuous-time Markov chains that describe the ...

Please sign up or login with your details

Forgot password? Click here to reset