Practical Estimation of Renyi Entropy
share
Entropy Estimation is an important problem with many applications in cryptography, statistic,machine learning. Although the estimators optimal with respect to the sample complexity have beenrecently developed, there are still some challenges we address in this paper.The contribution is a novel estimator which is built directly on the birthday paradox. Theanalysis turns out to be considerably simpler and offer superior confidence bounds with explicitconstants. We also discuss how streaming algorithm can be used to massively improve memoryconsumption. Last but not least, we study the problem of estimation in low or moderate regimes,adapting the estimator and proving rigorus bounds.
READ FULL TEXT