Classification with the nearest neighbor rule in general finite dimensional spaces: necessary and sufficient conditions

11/04/2014
by   Sébastien Gadat, et al.
0

Given an n-sample of random vectors (X_i,Y_i)_1 ≤ i ≤ n whose joint law is unknown, the long-standing problem of supervised classification aims to optimally predict the label Y of a given a new observation X. In this context, the nearest neighbor rule is a popular flexible and intuitive method in non-parametric situations. Even if this algorithm is commonly used in the machine learning and statistics communities, less is known about its prediction ability in general finite dimensional spaces, especially when the support of the density of the observations is R^d. This paper is devoted to the study of the statistical properties of the nearest neighbor rule in various situations. In particular, attention is paid to the marginal law of X, as well as the smoothness and margin properties of the regression function η(X) = E[Y | X]. We identify two necessary and sufficient conditions to obtain uniform consistency rates of classification and to derive sharp estimates in the case of the nearest neighbor rule. Some numerical experiments are proposed at the end of the paper to help illustrate the discussion.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/30/2014

Rates of Convergence for Nearest Neighbor Classification

Nearest neighbor methods are a popular class of nonparametric estimators...
research
07/25/2018

Topics in Random Matrices and Statistical Machine Learning

This thesis consists of two independent parts: random matrices, which fo...
research
07/03/2023

Online nearest neighbor classification

We study an instance of online non-parametric classification in the real...
research
02/25/2021

On the consistency of the Kozachenko-Leonenko entropy estimate

We revisit the problem of the estimation of the differential entropy H(f...
research
09/22/2019

Classification in asymmetric spaces via sample compression

We initiate the rigorous study of classification in quasi-metric spaces....
research
04/23/2018

Statistical Estimation of Conditional Shannon Entropy

The new estimates of the conditional Shannon entropy are introduced in t...
research
09/10/2020

Denoising modulo samples: k-NN regression and tightness of SDP relaxation

Many modern applications involve the acquisition of noisy modulo samples...

Please sign up or login with your details

Forgot password? Click here to reset