Optimal rates of entropy estimation over Lipschitz balls

11/06/2017
by   Yanjun Han, et al.
0

We consider the problem of minimax estimation of the entropy of a density over Lipschitz balls. Dropping the usual assumption that the density is bounded away from zero, we obtain the minimax rates (n n)^-s/s+d + n^-1/2 for 0<s≤ 2 in arbitrary dimension d, where s is the smoothness parameter and n is the number of independent samples. Using a two-stage approximation technique, which first approximate the density by its kernel-smoothed version, and then approximate the non-smooth functional by polynomials, we construct entropy estimators that attain the minimax rate of convergence, shown optimal by matching lower bounds. One of the key steps in analyzing the bias relies on a novel application of the Hardy-Littlewood maximal inequality, which also leads to a new inequality on Fisher information that might be of independent interest.

READ FULL TEXT

page 1

page 2

page 3

page 4

08/25/2020

Minimax estimation of norms of a probability density: II. Rate-optimal estimation procedures

In this paper we develop rate–optimal estimation procedures in the probl...
09/21/2021

Minimax Rates for Conditional Density Estimation via Empirical Entropy

We consider the task of estimating a conditional density using i.i.d. sa...
12/07/2021

Bless and curse of smoothness and phase transitions in nonparametric regressions: a nonasymptotic perspective

When the regression function belongs to the standard smooth classes cons...
01/07/2021

Density Deconvolution with Non-Standard Error Distributions: Rates of Convergence and Adaptive Estimation

It is a typical standard assumption in the density deconvolution problem...
09/09/2021

Goodness-of-Fit Testing for Hölder-Continuous Densities: Sharp Local Minimax Rates

We consider the goodness-of fit testing problem for Hölder smooth densit...
07/18/2020

Maximal-entropy driven determination of weights in least-square approximation

We exploit the idea to use the maximal-entropy method, successfully test...
06/15/2020

Estimation of Skill Distributions

In this paper, we study the problem of learning the skill distribution o...