Mini-batch learning of exponential family finite mixture models

02/09/2019
by   H D Nguyen, et al.
0

Mini-batch algorithms have become increasingly popular due to the requirement for solving optimization problems, based on large-scale data sets. Using an existing online expectation--maximization (EM) algorithm framework, we demonstrate how mini-batch (EM) algorithms may be constructed, and propose a scheme for the stochastic stabilization of the constructed mini-batch algorithms. Theoretical results regarding the convergence of these mini-batch EM algorithms are presented. We then demonstrate how the mini-batch framework may be applied to conduct maximum likelihood (ML) estimation of mixtures of exponential family distributions, with emphasis on ML estimation for mixtures of normal distributions. Via a simulation study, we demonstrate that the mini-batch algorithm for mixtures of normal distributions can outperform the standard EM algorithm. Further evidence of the performance of the mini-batch framework is provided via an application to the famous MNIST data set.

READ FULL TEXT
research
07/22/2019

Properties of the Stochastic Approximation EM Algorithm with Mini-batch Sampling

To speed up convergence a mini-batch version of the Monte Carlo Markov C...
research
09/17/2018

Homogeneity testing under finite location-scale mixtures

The testing problem for the order of finite mixture models has a long hi...
research
06/17/2020

Family of mean-mixtures of multivariate normal distributions: properties, inference and assessment of multivariate skewness

In this paper, a new mixture family of multivariate normal distributions...
research
03/02/2021

Mini-Batch Consistent Slot Set Encoder for Scalable Set Encoding

Most existing set encoding algorithms operate under the assumption that ...
research
07/26/2019

Unsupervised Learning Framework of Interest Point Via Properties Optimization

This paper presents an entirely unsupervised interest point training fra...
research
06/05/2023

LibAUC: A Deep Learning Library for X-Risk Optimization

This paper introduces the award-winning deep learning (DL) library calle...
research
07/20/2021

A Stochastic Version of the EM Algorithm for Mixture Cure Rate Model with Exponentiated Weibull Family of Lifetimes

Handling missing values plays an important role in the analysis of survi...

Please sign up or login with your details

Forgot password? Click here to reset