Using the Naive Bayes as a discriminative classifier

by   Elie Azeraf, et al.

For classification tasks, probabilistic models can be categorized into two disjoint classes: generative or discriminative. It depends on the posterior probability computation of the label x given the observation y, p(x | y). On the one hand, generative classifiers, like the Naive Bayes or the Hidden Markov Model (HMM), need the computation of the joint probability p(x, y), before using the Bayes rule to compute p(x | y). On the other hand, discriminative classifiers compute p(x | y) directly, regardless of the observations' law. They are intensively used nowadays, with models as Logistic Regression, Conditional Random Fields (CRF), and Artificial Neural Networks. However, the recent Entropic Forward-Backward algorithm shows that the HMM, considered as a generative model, can also match the discriminative one's definition. This example leads to question if it is the case for other generative models. In this paper, we show that the Naive Bayes classifier can also match the discriminative classifier definition, so it can be used in either a generative or a discriminative way. Moreover, this observation also discusses the notion of Generative-Discriminative pairs, linking, for example, Naive Bayes and Logistic Regression, or HMM and CRF. Related to this point, we show that the Logistic Regression can be viewed as a particular case of the Naive Bayes used in a discriminative way.


page 1

page 2

page 3

page 4


Deriving discriminative classifiers from generative models

We deal with Bayesian generative and discriminative classifiers. Given a...

The Mysterious Optimality of Naive Bayes: Estimation of the Probability in the System of "Classifiers"

Bayes Classifiers are widely used currently for recognition, identificat...

Are Generative Classifiers More Robust to Adversarial Attacks?

There is a rising interest in studying the robustness of deep neural net...

Deep clustering: On the link between discriminative models and K-means

In the context of recent deep clustering studies, discriminative models ...

Classification of Cuisines from Sequentially Structured Recipes

Cultures across the world are distinguished by the idiosyncratic pattern...

Entropy methods for the confidence assessment of probabilistic classification models

Many classification models produce a probability distribution as the out...

On equivalence between linear-chain conditional random fields and hidden Markov chains

Practitioners successfully use hidden Markov chains (HMCs) in different ...