kNN

kNN

The k-nearest neighbors algorithm, or kNN, is one of the simplest machine learning algorithms. Usually, k is a small, odd number - sometimes only 1. The larger k is, the more accurate the classification will be, but the longer it takes to perform the classification.

A Definition Expansion

Let’s say you want to classify an object into one of several classes -- for example, "pictures containing a face" and "pictures not containing a face". You do this by looking at the k elements of the training set that are closest to the one you want to classify, and letting them vote by majority on what that object’s class should be. If two of your closest elements were in class A and only one in class B, and k = 3, then you would conclude the element that are you trying to classify would go in class A. "Closest" here refers to literal distance in n-dimensional space, or the Euclidean distance.


There's also something called weighted kNN, which is like kNN except neighbors that are closer count as stronger votes. If there is one example of class A, and two examples of class B that are farther away, the algorithm still might classify the input as class A.

Application in AI

kNN is a family of machine learning algorithms, and is among some of the simplest.