Log In Sign Up

A Tutorial on the Spectral Theory of Markov Chains

by   Eddie Seabrook, et al.

Markov chains are a class of probabilistic models that have achieved widespread application in the quantitative sciences. This is in part due to their versatility, but is compounded by the ease with which they can be probed analytically. This tutorial provides an in-depth introduction to Markov chains, and explores their connection to graphs and random walks. We utilize tools from linear algebra and graph theory to describe the transition matrices of different types of Markov chains, with a particular focus on exploring properties of the eigenvalues and eigenvectors corresponding to these matrices. The results presented are relevant to a number of methods in machine learning and data mining, which we describe at various stages. Rather than being a novel academic study in its own right, this text presents a collection of known results, together with some new concepts. Moreover, the tutorial focuses on offering intuition to readers rather than formal understanding, and only assumes basic exposure to concepts from linear algebra and probability theory. It is therefore accessible to students and researchers from a wide variety of disciplines.


page 20

page 34


Moderate deviations for empirical measures for nonhomogeneous Markov chains

We prove that moderate deviations for empirical measures for countable n...

Eigenvalues and Eigenvectors of Tau Matrices with Applications to Markov Processes and Economics

In the context of matrix displacement decomposition, Bozzo and Di Fiore ...

Approximate quantum Markov chains

This book is an introduction to quantum Markov chains and explains how t...

Markov chains in random environment with applications in queueing theory and machine learning

We prove the existence of limiting distributions for a large class of Ma...

Learning Multiple Markov Chains via Adaptive Allocation

We study the problem of learning the transition matrices of a set of Mar...

Rational Krylov and ADI iteration for infinite size quasi-Toeplitz matrix equations

We consider a class of linear matrix equations involving semi-infinite m...

Rewriting Theory for the Life Sciences: A Unifying Theory of CTMC Semantics

The Kappa biochemistry and the MØD organo-chemistry frameworks are among...