The f-Divergence Expectation Iteration Scheme
share
This paper introduces the f-EI(ϕ) algorithm, a novel iterative algorithm which operates on measures and performs f-divergence minimisation in a Bayesian framework. We prove that for a rich family of values of (f,ϕ) this algorithm leads at each step to a systematic decrease in the f-divergence and show that we achieve an optimum. In the particular case where we consider a weighted sum of Dirac measures and the α-divergence, we obtain that the calculations involved in the f-EI(ϕ) algorithm simplify to gradient-based computations. Empirical results support the claim that the f-EI(ϕ) algorithm serves as a powerful tool to assist Variational methods.
READ FULL TEXT