Fast Maximum Likelihood Estimation and Supervised Classification for the Beta-Liouville Multinomial

by   Steven Michael Lakin, et al.

The multinomial and related distributions have long been used to model categorical, count-based data in fields ranging from bioinformatics to natural language processing. Commonly utilized variants include the standard multinomial and the Dirichlet multinomial distributions due to their computational efficiency and straightforward parameter estimation process. However, these distributions make strict assumptions about the mean, variance, and covariance between the categorical features being modeled. If these assumptions are not met by the data, it may result in poor parameter estimates and loss in accuracy for downstream applications like classification. Here, we explore efficient parameter estimation and supervised classification methods using an alternative distribution, called the Beta-Liouville multinomial, which relaxes some of the multinomial assumptions. We show that the Beta-Liouville multinomial is comparable in efficiency to the Dirichlet multinomial for Newton-Raphson maximum likelihood estimation, and that its performance on simulated data matches or exceeds that of the multinomial and Dirichlet multinomial distributions. Finally, we demonstrate that the Beta-Liouville multinomial outperforms the multinomial and Dirichlet multinomial on two out of four gold standard datasets, supporting its use in modeling data with low to medium class overlap in a supervised classification context.



There are no comments yet.


page 1

page 2

page 3

page 4


Estimation of Dirichlet distribution parameters with bias-reducing adjusted score functions

The Dirichlet distribution, also known as multivariate beta, is the most...

Accurate parameter estimation for Bayesian Network Classifiers using Hierarchical Dirichlet Processes

This paper introduces a novel parameter estimation method for the probab...

A Conway-Maxwell-Multinomial Distribution for Flexible Modeling of Clustered Categorical Data

Categorical data are often observed as counts resulting from a fixed num...

Developing multivariate distributions using Dirichlet generator

There exist several endeavors proposing a new family of extended distrib...

MedLDA: A General Framework of Maximum Margin Supervised Topic Models

Supervised topic models utilize document's side information for discover...

Fast MLE Computation for the Dirichlet Multinomial

Given a collection of categorical data, we want to find the parameters o...

The continuous categorical: a novel simplex-valued exponential family

Simplex-valued data appear throughout statistics and machine learning, f...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.