Structured and Efficient Variational Deep Learning with Matrix Gaussian Posteriors

03/15/2016
by   Christos Louizos, et al.
0

We introduce a variational Bayesian neural network where the parameters are governed via a probability distribution on random matrices. Specifically, we employ a matrix variate Gaussian gupta1999matrix parameter posterior distribution where we explicitly model the covariance among the input and output dimensions of each layer. Furthermore, with approximate covariance matrices we can achieve a more efficient way to represent those correlations that is also cheaper than fully factorized parameter posteriors. We further show that with the "local reprarametrization trick" kingma2015variational on this posterior distribution we arrive at a Gaussian Process rasmussen2006gaussian interpretation of the hidden units in each layer and we, similarly with gal2015dropout, provide connections with deep Gaussian processes. We continue in taking advantage of this duality and incorporate "pseudo-data" snelson2005sparse in our model, which in turn allows for more efficient sampling while maintaining the properties of the original model. The validity of the proposed approach is verified through extensive experiments.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/17/2019

Compositional uncertainty in deep Gaussian processes

Gaussian processes (GPs) are nonparametric priors over functions, and fi...
research
05/23/2023

An Improved Variational Approximate Posterior for the Deep Wishart Process

Deep kernel processes are a recently introduced class of deep Bayesian m...
research
10/05/2020

Bayesian Fixed-domain Asymptotics: Bernstein-von Mises Theorem for Covariance Parameters in a Gaussian Process Model

Gaussian process models typically contain finite dimensional parameters ...
research
11/16/2018

A multilayer exponential random graph modelling approach for weighted networks

This paper introduces a new modelling approach to analyse weighted netwo...
research
04/02/2019

Correlated Parameters to Accurately Measure Uncertainty in Deep Neural Networks

In this article a novel approach for training deep neural networks using...
research
01/22/2015

Enabling scalable stochastic gradient-based inference for Gaussian processes by employing the Unbiased LInear System SolvEr (ULISSE)

In applications of Gaussian processes where quantification of uncertaint...
research
03/29/2022

Learning Structured Gaussians to Approximate Deep Ensembles

This paper proposes using a sparse-structured multivariate Gaussian to p...

Please sign up or login with your details

Forgot password? Click here to reset