Efficient convergence through adaptive learning in sequential Monte Carlo Expectation Maximization

07/09/2018
by   Donna Henderson, et al.
0

Expectation maximization (EM) is a technique for estimating maximum-likelihood parameters of a latent variable model given observed data by alternating between taking expectations of sufficient statistics, and maximizing the expected log likelihood. For situations where sufficient statistics are intractable, stochastic approximation EM (SAEM) is often used, which uses Monte Carlo techniques to approximate the expected log likelihood. Two common implementations of SAEM, Batch EM (BEM) and online EM (OEM), are parameterized by a "learning rate", and their efficiency depend strongly on this parameter. We propose an extension to the OEM algorithm, termed Introspective Online Expectation Maximization (IOEM), which removes the need for specifying this parameter by adapting the learning rate according to trends in the parameter updates. We show that our algorithm matches the efficiency of the optimal BEM and OEM algorithms in multiple models, and that the efficiency of IOEM can exceed that of BEM/OEM methods with optimal learning rates when the model has many parameters. A Python implementation is available at https://github.com/luntergroup/IOEM.git.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/21/2022

EM's Convergence in Gaussian Latent Tree Models

We study the optimization landscape of the log-likelihood function and t...
research
02/11/2019

Divergence-Based Motivation for Online EM and Combining Hidden Variable Models

Expectation-Maximization (EM) is the fallback method for parameter estim...
research
06/05/2022

RPEM: Randomized Monte Carlo Parametric Expectation Maximization Algorithm

Inspired from quantum Monte Carlo methods, we developed a novel, fast, a...
research
06/05/2021

Parameter Estimation for Grouped Data Using EM and MCEM Algorithms

Nowadays, the confidentiality of data and information is of great import...
research
09/07/2016

Learning Boltzmann Machine with EM-like Method

We propose an expectation-maximization-like(EMlike) method to train Bolt...
research
06/20/2020

Scalable Identification of Partially Observed Systems with Certainty-Equivalent EM

System identification is a key step for model-based control, estimator d...
research
05/15/2019

A New Estimation Algorithm for Box-Cox Transformation Cure Rate Model and Comparison With EM Algorithm

In this paper, we develop a new estimation procedure based on the non-li...

Please sign up or login with your details

Forgot password? Click here to reset