Bayesian Hypernetworks

10/13/2017
by   David Krueger, et al.
0

We propose Bayesian hypernetworks: a framework for approximate Bayesian inference in neural networks. A Bayesian hypernetwork, h, is a neural network which learns to transform a simple noise distribution, p(ϵ) = N(0,I), to a distribution q(θ) q(h(ϵ)) over the parameters θ of another neural network (the "primary network"). We train q with variational inference, using an invertible h to enable efficient estimation of the variational lower bound on the posterior p(θ | D) via sampling. In contrast to most methods for Bayesian deep learning, Bayesian hypernets can represent a complex multimodal approximate posterior with correlations between parameters, while enabling cheap i.i.d. sampling of q(θ). We demonstrate these qualitative advantages of Bayesian hypernets, which also achieve competitive performance on a suite of tasks that demonstrate the advantage of estimating model uncertainty, including active learning and anomaly detection.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/27/2020

Stochastic Bayesian Neural Networks

Bayesian neural networks perform variational inference over weights but ...
research
03/20/2017

Efficient variational Bayesian neural network ensembles for outlier detection

In this work we perform outlier detection using ensembles of neural netw...
research
06/27/2018

Adversarial Distillation of Bayesian Neural Network Posteriors

Bayesian neural networks (BNNs) allow us to reason about uncertainty in ...
research
06/10/2019

Stochastic Neural Network with Kronecker Flow

Recent advances in variational inference enable the modelling of highly ...
research
12/31/2017

Using Deep Neural Network Approximate Bayesian Network

We present a new method to approximate posterior probabilities of Bayesi...
research
02/21/2018

A Generative Deep Recurrent Model for Exchangeable Data

We present a novel model architecture which leverages deep learning tool...
research
03/15/2021

Sampling-free Variational Inference for Neural Networks with Multiplicative Activation Noise

To adopt neural networks in safety critical domains, knowing whether we ...

Please sign up or login with your details

Forgot password? Click here to reset