Minimax Learning of Ergodic Markov Chains

09/13/2018
by   Geoffrey Wolfer, et al.
0

We compute the finite-sample minimax (modulo logarithmic factors) sample complexity of learning the parameters of a finite Markov chain from a single long sequence of states. Our error metric is a natural variant of total variation. The sample complexity necessarily depends on the spectral gap and minimal stationary probability of the unknown chain - for which, at least in the reversible case, there are known finite-sample estimators with fully empirical confidence intervals. To our knowledge, this is the first PAC-type result with nearly matching (up to logs) upper and lower bounds for learning, in any metric in the context of Markov chains.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/31/2019

Minimax Testing of Identity to a Reference Ergodic Markov Chain

We exhibit an efficient procedure for testing, based on a single long st...
research
11/14/2022

Offline Estimation of Controlled Markov Chains: Minimax Nonparametric Estimators and Sample Efficiency

Controlled Markov chains (CMCs) form the bedrock for model-based reinfor...
research
02/01/2019

Estimating the Mixing Time of Ergodic Markov Chains

We address the problem of estimating the mixing time t_mix of an arbitra...
research
08/06/2018

Statistical Windows in Testing for the Initial Distribution of a Reversible Markov Chain

We study the problem of hypothesis testing between two discrete distribu...
research
10/20/2018

Learning Models with Uniform Performance via Distributionally Robust Optimization

A common goal in statistics and machine learning is to learn models that...
research
08/24/2017

Mixing time estimation in reversible Markov chains from a single sample path

The spectral gap γ of a finite, ergodic, and reversible Markov chain is ...
research
10/08/2021

Learning from non-irreducible Markov chains

Most of the existing literature on supervised learning problems focuses ...

Please sign up or login with your details

Forgot password? Click here to reset