Efficient Gradient-Based Inference through Transformations between Bayes Nets and Neural Nets

02/03/2014
by   Diederik P. Kingma, et al.
0

Hierarchical Bayesian networks and neural networks with stochastic hidden units are commonly perceived as two separate types of models. We show that either of these types of models can often be transformed into an instance of the other, by switching between centered and differentiable non-centered parameterizations of the latent variables. The choice of parameterization greatly influences the efficiency of gradient-based posterior inference; we show that they are often complementary to eachother, we clarify when each parameterization is preferred and show how inference can be made robust. In the non-centered form, a simple Monte Carlo estimator of the marginal likelihood can be used for learning the parameters. Theoretical results are supported by experiments.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/04/2013

Fast Gradient-Based Inference with Continuous Latent Variable Models in Auxiliary Form

We propose a technique for increasing the efficiency of gradient-based i...
research
11/19/2022

Non-centered parametric variational Bayes' approach for hierarchical inverse problems of partial differential equations

This paper proposes a non-centered parameterization based infinite-dimen...
research
12/19/2018

A Note on Lazy Training in Supervised Differentiable Programming

In a series of recent theoretical works, it has been shown that strongly...
research
06/11/2020

Latent Transformations for Discrete-Data Normalising Flows

Normalising flows (NFs) for discrete data are challenging because parame...
research
05/19/2018

Sampling-Free Variational Inference of Bayesian Neural Nets

We propose a new Bayesian Neural Net (BNN) formulation that affords vari...
research
01/10/2013

Inference in Hybrid Networks: Theoretical Limits and Practical Algorithms

An important subclass of hybrid Bayesian networks are those that represe...
research
02/12/2022

Adaptive truncation of infinite sums: applications to Statistics

It is often the case in Statistics that one needs to compute sums of inf...

Please sign up or login with your details

Forgot password? Click here to reset