Monotonic Alpha-divergence Minimisation

03/09/2021
by   Kamélia Daudel, et al.
0

In this paper, we introduce a novel iterative algorithm which carries out α-divergence minimisation by ensuring a systematic decrease in the α-divergence at each step. In its most general form, our framework allows us to simultaneously optimise the weights and components parameters of a given mixture model. Notably, our approach permits to build on various methods previously proposed for α-divergence minimisation such as gradient or power descent schemes. Furthermore, we shed a new light on an integrated Expectation Maximization algorithm. We provide empirical evidence that our methodology yields improved results, all the while illustrating the numerical benefits of having introduced some flexibility through the parameter α of the α-divergence.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/20/2020

Infinite-dimensional gradient-based descent for alpha-divergence minimisation

This paper introduces the (α, Γ)-descent, an iterative algorithm which o...
research
06/09/2021

Mixture weights optimisation for Alpha-Divergence Variational Inference

This paper focuses on α-divergence minimisation methods for Variational ...
research
09/26/2019

The f-Divergence Expectation Iteration Scheme

This paper introduces the f-EI(ϕ) algorithm, a novel iterative algorithm...
research
06/13/2021

Adaptation of the Tuning Parameter in General Bayesian Inference with Robust Divergence

We introduce a methodology for robust Bayesian estimation with robust di...
research
01/07/2022

Bregman divergence based em algorithm and its application to classical and quantum rate distortion theory

We formulate em algorithm in the framework of Bregman divergence, which ...
research
08/01/2023

Divergence of the ADAM algorithm with fixed-stepsize: a (very) simple example

A very simple unidimensional function with Lipschitz continuous gradient...
research
04/25/2019

Uniform minimum risk equivariant estimates for moment condition models

We consider semiparametric moment condition models invariant to transfor...

Please sign up or login with your details

Forgot password? Click here to reset