Entropy Rate Estimation for Markov Chains with Large State Space

02/22/2018
by   Yanjun Han, et al.
0

Estimating the entropy based on data is one of the prototypical problems in distribution property testing and estimation. For estimating the Shannon entropy of a distribution on S elements with independent samples, [Paninski2004] showed that the sample complexity is sublinear in S, and [Valiant--Valiant2011] showed that consistent estimation of Shannon entropy is possible if and only if the sample size n far exceeds S/ S. In this paper we consider the problem of estimating the entropy rate of a stationary reversible Markov chain with S states from a sample path of n observations. We show that: (1) As long as the Markov chain mixes not too slowly, i.e., the relaxation time is at most O(S/^3 S), consistent estimation is achievable when n ≫S^2/ S. (2) As long as the Markov chain has some slight dependency, i.e., the relaxation time is at least 1+Ω(^2 S/√(S)), consistent estimation is impossible when n ≲S^2/ S. Under both assumptions, the optimal estimation accuracy is shown to be Θ(S^2/n S). In comparison, the empirical entropy rate requires at least Ω(S^2) samples to be consistent, even when the Markov chain is memoryless. In addition to synthetic experiments, we also apply the estimators that achieve the optimal sample complexity to estimate the entropy rate of the English language in the Penn Treebank and the Google One Billion Words corpora, which provides a natural benchmark for language modeling and relates it directly to the widely used perplexity measure.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/16/2023

A Geometric Reduction Approach for Identity Testing of Reversible Markov Chains

We consider the problem of testing the identity of a reversible Markov c...
research
08/06/2018

Statistical Windows in Testing for the Initial Distribution of a Reversible Markov Chain

We study the problem of hypothesis testing between two discrete distribu...
research
10/28/2021

Convergence of Conditional Entropy for Long Range Dependent Markov Chains

In this paper we consider the convergence of the conditional entropy to ...
research
11/10/2017

Estimating the Entropy Rate of Finite Markov Chains with Application to Behavior Studies

Predictability of behavior has emerged an an important characteristic in...
research
09/23/2020

Estimating entropy rate from censored symbolic time series: a test for time-irreversibility

In this work we introduce a method for estimating entropy rate and entro...
research
11/18/2019

Estimating Entropy of Distributions in Constant Space

We consider the task of estimating the entropy of k-ary distributions fr...
research
11/23/2022

Perfect Sampling from Pairwise Comparisons

In this work, we study how to efficiently obtain perfect samples from a ...

Please sign up or login with your details

Forgot password? Click here to reset