Analysis of KNN Density Estimation

by   Puning Zhao, et al.

We analyze the ℓ_1 and ℓ_∞ convergence rates of k nearest neighbor density estimation method. Our analysis includes two different cases depending on whether the support set is bounded or not. In the first case, the probability density function has a bounded support and is bounded away from zero. We show that kNN density estimation is minimax optimal under both ℓ_1 and ℓ_∞ criteria, if the support set is known. If the support set is unknown, then the convergence rate of ℓ_1 error is not affected, while ℓ_∞ error does not converge. In the second case, the probability density function can approach zero and is smooth everywhere. Moreover, the Hessian is assumed to decay with the density values. For this case, our result shows that the ℓ_∞ error of kNN density estimation is nearly minimax optimal. The ℓ_1 error does not reach the minimax lower bound, but is better than kernel density estimation.



page 1

page 2

page 3

page 4


Minimax Rate Optimal Adaptive Nearest Neighbor Classification and Regression

k Nearest Neighbor (kNN) method is a simple and popular statistical meth...

Asymptotic nonequivalence of density estimation and Gaussian white noise for small densities

It is well-known that density estimation on the unit interval is asympto...

Clustering by Deep Nearest Neighbor Descent (D-NND): A Density-based Parameter-Insensitive Clustering Method

Most density-based clustering methods largely rely on how well the under...

Minimax Rates for Conditional Density Estimation via Empirical Entropy

We consider the task of estimating a conditional density using i.i.d. sa...

Fused Density Estimation: Theory and Methods

In this paper we introduce a method for nonparametric density estimation...

Bayesian Point Estimation and Predictive Density Estimation for the Binomial Distribution with a Restricted Probability Parameter

In this paper, we consider Bayesian point estimation and predictive dens...

Optimal rates of entropy estimation over Lipschitz balls

We consider the problem of minimax estimation of the entropy of a densit...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.