Log In Sign Up

An Efficient Algorithm for Bayesian Nearest Neighbours

by   Giuseppe Nuti, et al.

K-Nearest Neighbours (k-NN) is a popular classification and regression algorithm, yet one of its main limitations is the difficulty in choosing the number of neighbours. We present a Bayesian algorithm to compute the posterior probability distribution for k given a target point within a data-set, efficiently and without the use of Markov Chain Monte Carlo (MCMC) methods or simulation - alongside an exact solution for distributions within the exponential family. The central idea is that data points around our target are generated by the same probability distribution, extending outwards over the appropriate, though unknown, number of neighbours. Once the data is projected onto a distance metric of choice, we can transform the choice of k into a change-point detection problem, for which there is an efficient solution: we recursively compute the probability of the last change-point as we move towards our target, and thus de facto compute the posterior probability distribution over k. Applying this approach to both a classification and a regression UCI data-sets, we compare favourably and, most importantly, by removing the need for simulation, we are able to compute the posterior probability of k exactly and rapidly. As an example, the computational time for the Ripley data-set is a few milliseconds compared to a few hours when using a MCMC approach.


page 1

page 2

page 3

page 4


Change-point Detection for Piecewise Exponential Models

In decision modelling with time to event data, parametric models are oft...

A Predictive Model using the Markov Property

Given a data set of numerical values which are sampled from some unknown...

Being Bayesian about Network Structure

In many domains, we are interested in analyzing the structure of the und...

The Universal model and prior: multinomial GLMs

This paper generalises the exponential family GLM to allow arbitrary dis...

A more efficient algorithm to compute the Rand Index for change-point problems

In this paper we provide a more efficient algorithm to compute the Rand ...

Is the NUTS algorithm correct?

This paper is devoted to investigate whether the popular No U-turn (NUTS...

Bayesian models in geographic profiling

We consider the problem of geographic profiling and offer an approach to...