On the consistency of the Kozachenko-Leonenko entropy estimate

02/25/2021

∙

by   Luc Devroye, et al.

∙

0

∙

share

We revisit the problem of the estimation of the differential entropy H(f) of a random vector X in R^d with density f, assuming that H(f) exists and is finite. In this note, we study the consistency of the popular nearest neighbor estimate H_n of Kozachenko and Leonenko. Without any smoothness condition we show that the estimate is consistent (E{|H_n - H(f)|}→ 0 as n →∞) if and only if 𝔼{log ( X + 1 )} < ∞. Furthermore, if X has compact support, then H_n → H(f) almost surely.