Optimal Chernoff and Hoeffding Bounds for Finite Markov Chains

07/10/2019
by   Vrettos Moulos, et al.
0

This paper develops an optimal Chernoff type bound for the probabilities of large deviations of sums ∑_k=1^n f (X_k) where f is a real-valued function and (X_k)_k ∈N_0 is a finite Markov chain with an arbitrary initial distribution and an irreducible stochastic matrix coming from a large class of stochastic matrices. Our bound is optimal in the large deviations sense attaining a constant prefactor and an exponential decay with the optimal large deviations rate. Moreover through a Pinsker type inequality and a Hoeffding type lemma, we are able to loosen up our Chernoff type bound to a Hoeffding type bound and reveal the sub-Gaussian nature of the sums. Finally we show a uniform multiplicative ergodic theorem for our class of Markov chains.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/10/2021

Optimal bounds for bit-sizes of stationary distributions in finite Markov chains

An irreducible stochastic matrix with rational entries has a stationary ...
research
01/05/2020

A Hoeffding Inequality for Finite State Markov Chains and its Applications to Markovian Bandits

This paper develops a Hoeffding inequality for the partial sums ∑_k=1^n ...
research
10/11/2019

Iterated Decomposition of Biased Permutations Via New Bounds on the Spectral Gap of Markov Chains

The spectral gap of a Markov chain can be bounded by the spectral gaps o...
research
05/20/2016

Quantifying the accuracy of approximate diffusions and Markov chains

Markov chains and diffusion processes are indispensable tools in machine...
research
04/02/2022

Various proofs of the Fundamental Theorem of Markov Chains

This paper is a survey of various proofs of the so called fundamental th...
research
07/27/2018

On the Inability of Markov Models to Capture Criticality in Human Mobility

We examine the non-Markovian nature of human mobility by exposing the in...
research
09/28/2020

Decisiveness of Stochastic Systems and its Application to Hybrid Models (Full Version)

In [ABM07], Abdulla et al. introduced the concept of decisiveness, an in...

Please sign up or login with your details

Forgot password? Click here to reset