DeepAI AI Chat
Log In Sign Up

Hoeffding's lemma for Markov Chains and its applications to statistical learning

by   Jianqing Fan, et al.

We establish the counterpart of Hoeffding's lemma for Markov dependent random variables. Specifically, if a stationary Markov chain {X_i}_i > 1 with invariant measure π admits an L_2(π)-spectral gap 1-λ, then for any bounded functions f_i: x [a_i,b_i], the sum of f_i(X_i) is sub-Gaussian with variance proxy 1+λ/1-λ·∑_i (b_i-a_i)^2/4. The counterpart of Hoeffding's inequality immediately follows. Our results assume none of reversibility, countable state space and time-homogeneity of Markov chains. They are optimal in terms of the multiplicative coefficient (1+λ)/(1-λ), and cover Hoeffding's lemma and inequality for independent random variables as special cases with λ = 0. We illustrate the utility of these results by applying them to six problems in statistics and machine learning. They are linear regression, lasso regression, sparse covariance matrix estimation with Markov-dependent samples; Markov chain Monte Carlo estimation; respondence driven sampling; and multi-armed bandit problems with Markovian rewards.


page 1

page 2

page 3

page 4


Bernstein's inequality for general Markov chains

We prove a sharp Bernstein inequality for general-state-space and not ne...

A Hoeffding Inequality for Finite State Markov Chains and its Applications to Markovian Bandits

This paper develops a Hoeffding inequality for the partial sums ∑_k=1^n ...

Concentration inequality for U-statistics of order two for uniformly ergodic Markov chains, and applications

We prove a new concentration inequality for U-statistics of order two fo...

Three rates of convergence or separation via U-statistics in a dependent framework

Despite the ubiquity of U-statistics in modern Probability and Statistic...

Concentration without Independence via Information Measures

We propose a novel approach to concentration for non-independent random ...

Regularized Modal Regression on Markov-dependent Observations: A Theoretical Assessment

Modal regression, a widely used regression protocol, has been extensivel...

On Markov chain Monte Carlo for sparse and filamentary distributions

A novel strategy that combines a given collection of reversible Markov k...