Concentration Inequalities for the Empirical Distribution

09/18/2018

∙

by   Jay Mardia, et al.

∙

0

∙

share

We study concentration inequalities for the Kullback--Leibler (KL) divergence between the empirical distribution and the true distribution. Applying a recursion technique, we improve over the method of types bound uniformly in all regimes of sample size n and alphabet size k, and the improvement becomes more significant when k is large. We discuss the applications of our results in obtaining tighter concentration inequalities for L_1 deviations of the empirical distribution from the true distribution, and the difference between concentration around the expectation or zero.