Assessing the Robustness of Bayesian Dark Knowledge to Posterior Uncertainty

06/04/2019
by   Meet P. Vadera, et al.
0

Bayesian Dark Knowledge is a method for compressing the posterior predictive distribution of a neural network model into a more compact form. Specifically, the method attempts to compress a Monte Carlo approximation to the parameter posterior into a single network representing the posterior predictive distribution. Further, the authors show that this approach is successful in the classification setting using a student network whose architecture matches that of a single network in the teacher ensemble. In this work, we examine the robustness of Bayesian Dark Knowledge to higher levels of posterior uncertainty. We show that using a student network that matches the teacher architecture may fail to yield acceptable performance. We study an approach to close the resulting performance gap by increasing student model capacity.

READ FULL TEXT
research
05/16/2020

Generalized Bayesian Posterior Expectation Distillation for Deep Neural Networks

In this paper, we present a general framework for distilling expectation...
research
06/05/2023

Gibbs Sampling the Posterior of Neural Networks

In this paper, we study sampling from a posterior derived from a neural ...
research
05/29/2019

Accelerating Monte Carlo Bayesian Inference via Approximating Predictive Uncertainty over Simplex

Estimating the uncertainty of a Bayesian model has been investigated for...
research
12/28/2021

A Bayesian network model for predicting cardiovascular risk

We propose a Bayesian network model to make inferences and predictions a...
research
06/14/2015

Bayesian Dark Knowledge

We consider the problem of Bayesian parameter estimation for deep neural...
research
01/15/2023

A Simple Proof of Posterior Robustness

Conditions for Bayesian posterior robustness have been examined in recen...
research
03/20/2013

Reasoning under Uncertainty: Some Monte Carlo Results

A series of monte carlo studies were performed to compare the behavior o...

Please sign up or login with your details

Forgot password? Click here to reset