k*-Nearest Neighbors: From Global to Local

01/25/2017
by   Oren Anava, et al.
0

The weighted k-nearest neighbors algorithm is one of the most fundamental non-parametric methods in pattern recognition and machine learning. The question of setting the optimal number of neighbors as well as the optimal weights has received much attention throughout the years, nevertheless this problem seems to have remained unsettled. In this paper we offer a simple approach to locally weighted regression/classification, where we make the bias-variance tradeoff explicit. Our formulation enables us to phrase a notion of optimal weights, and to efficiently find these weights as well as the optimal number of neighbors efficiently and adaptively, for each data point whose value we wish to estimate. The applicability of our approach is demonstrated on several datasets, showing superior performance over standard locally weighted methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/01/2019

Active Search for Nearest Neighbors

In pattern recognition or machine learning, it is a very fundamental tas...
research
01/27/2013

An improvement to k-nearest neighbor classifier

K-Nearest neighbor classifier (k-NNC) is simple to use and has little de...
research
11/08/2020

Locally Adaptive Nearest Neighbors

When training automated systems, it has been shown to be beneficial to a...
research
03/26/2021

Applying k-nearest neighbors to time series forecasting : two new approaches

K-nearest neighbors algorithm is one of the prominent techniques used in...
research
04/09/2020

Multiclass Classification via Class-Weighted Nearest Neighbors

We study statistical properties of the k-nearest neighbors algorithm for...
research
11/12/2022

Far Away in the Deep Space: Nearest-Neighbor-Based Dense Out-of-Distribution Detection

The key to out-of-distribution detection is density estimation of the in...
research
02/18/2021

Consistent Non-Parametric Methods for Adaptive Robustness

Learning classifiers that are robust to adversarial examples has receive...

Please sign up or login with your details

Forgot password? Click here to reset