Optimal rates of entropy estimation over Lipschitz balls

by   Yanjun Han, et al.

We consider the problem of minimax estimation of the entropy of a density over Lipschitz balls. Dropping the usual assumption that the density is bounded away from zero, we obtain the minimax rates (n n)^-s/s+d + n^-1/2 for 0<s≤ 2 in arbitrary dimension d, where s is the smoothness parameter and n is the number of independent samples. Using a two-stage approximation technique, which first approximate the density by its kernel-smoothed version, and then approximate the non-smooth functional by polynomials, we construct entropy estimators that attain the minimax rate of convergence, shown optimal by matching lower bounds. One of the key steps in analyzing the bias relies on a novel application of the Hardy-Littlewood maximal inequality, which also leads to a new inequality on Fisher information that might be of independent interest.


page 1

page 2

page 3

page 4


Minimax estimation of norms of a probability density: II. Rate-optimal estimation procedures

In this paper we develop rate–optimal estimation procedures in the probl...

Minimax Rates for Conditional Density Estimation via Empirical Entropy

We consider the task of estimating a conditional density using i.i.d. sa...

Bless and curse of smoothness and phase transitions in nonparametric regressions: a nonasymptotic perspective

When the regression function belongs to the standard smooth classes cons...

Density Deconvolution with Non-Standard Error Distributions: Rates of Convergence and Adaptive Estimation

It is a typical standard assumption in the density deconvolution problem...

Goodness-of-Fit Testing for Hölder-Continuous Densities: Sharp Local Minimax Rates

We consider the goodness-of fit testing problem for Hölder smooth densit...

Maximal-entropy driven determination of weights in least-square approximation

We exploit the idea to use the maximal-entropy method, successfully test...

Estimation of Skill Distributions

In this paper, we study the problem of learning the skill distribution o...