Naive Bayes

What are Naive Bayes Classifiers?

Naive Bayes classifiers are a family of very simple classifiers based on the application of Bayes Theorem. Although they're relatively simple to implement, they work surprisingly well on many real-world problems. In addition, they have the advantage of scaling linearly in time complexity with the number of features, which means that they remain a useful tool for high-dimensional data, whereas other kinds of classifiers are unable to cope with the so-called curse of dimensionality.

One drawback of Naive Bayes classifiers is the reason that they're called naive: they assume that all the features are independent of each other, which usually isn't true for real-world data. This can lead a Naive Bayes classifier to make serious errors in its internal probability model distribution. Despite this drawback, Naive Bayes classifiers exhibit surprising accuracy considering the simplicity of their algorithms, and enjoy use in a variety of applications, such as spam filtering -- indeed, Naive Bayes classification is one of the oldest spam filtering techniques, dating back to the 1990's.