kNN

The k-nearest neighbors algorithm, or kNN, is one of the simplest machine learning algorithms. Usually, k is a small, odd number - sometimes only 1. The larger k is, the more accurate the classification will be, but the longer it takes to perform the classification.

Let’s say you want to classify an object into one of several classes -- for example, "pictures containing a face" and "pictures not containing a face". You do this by looking at the k elements of the training set that are closest to the one you want to classify, and letting them vote by majority on what that object’s class should be. If two of your closest elements were in class A and only one in class B, and k = 3, then you would conclude the element that are you trying to classify would go in class A. "Closest" here refers to literal distance in n-dimensional space, or the Euclidean distance.