Conditional Expectation Propagation

10/27/2019
by   Zheng Wang, et al.
0

Expectation propagation (EP) is a powerful approximate inference algorithm. However, a critical barrier in applying EP is that the moment matching in message updates can be intractable. Handcrafting approximations is usually tricky, and lacks generalizability. Importance sampling is very expensive. While Laplace propagation provides a good solution, it has to run numerical optimizations to find Laplace approximations in every update, which is still quite inefficient. To overcome these practical barriers, we propose conditional expectation propagation (CEP) that performs conditional moment matching given the variables outside each message, and then takes expectation w.r.t the approximate posterior of these variables. The conditional moments are often analytical and much easier to derive. In the most general case, we can use (fully) factorized messages to represent the conditional moments by quadrature formulas. We then compute the expectation of the conditional moments via Taylor approximations when necessary. In this way, our algorithm can always conduct efficient, analytical fixed point iterations. Experiments on several popular models for which standard EP is available or unavailable demonstrate the advantages of CEP in both inference quality and computational efficiency.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/01/2009

Expectation Propagation on the Maximum of Correlated Normal Variables

Many inference problems involving questions of optimality ask for the ma...
research
10/18/2017

On Least Squares Linear Regression Without Second Moment

If X and Y are real valued random variables such that the first moments ...
research
10/18/2018

Expectation Propagation for Poisson Data

The Poisson distribution arises naturally when dealing with data involvi...
research
01/02/2015

Passing Expectation Propagation Messages with Kernel Methods

We propose to learn a kernel-based message operator which takes as input...
research
04/18/2012

Message passing with relaxed moment matching

Bayesian learning is often hampered by large computational expense. As a...
research
01/12/2013

Perturbative Corrections for Approximate Inference in Gaussian Latent Variable Models

Expectation Propagation (EP) provides a framework for approximate infere...
research
03/01/2017

Semi-analytical approximations to statistical moments of sigmoid and softmax mappings of normal variables

This note is concerned with accurate and computationally efficient appro...

Please sign up or login with your details

Forgot password? Click here to reset