Implicit Priors for Knowledge Sharing in Bayesian Neural Networks

12/02/2019
by   Jack K. Fitzsimons, et al.
0

Bayesian interpretations of neural network have a long history, dating back to early work in the 1990's and have recently regained attention because of their desirable properties like uncertainty estimation, model robustness and regularisation. We want to discuss here the application of Bayesian models to knowledge sharing between neural networks. Knowledge sharing comes in different facets, such as transfer learning, model distillation and shared embeddings. All of these tasks have in common that learned "features" ought to be shared across different networks. Theoretically rooted in the concepts of Bayesian neural networks this work has widespread application to general deep learning.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/03/2019

On the Validity of Bayesian Neural Networks for Uncertainty Estimation

Deep neural networks (DNN) are versatile parametric models utilised succ...
research
07/15/2023

Learning Expressive Priors for Generalization and Uncertainty Estimation in Neural Networks

In this work, we propose a novel prior learning method for advancing gen...
research
06/18/2020

Shapeshifter Networks: Cross-layer Parameter Sharing for Scalable and Effective Deep Learning

We present Shapeshifter Networks (SSNs), a flexible neural network frame...
research
12/01/2017

Propagating Uncertainty in Multi-Stage Bayesian Convolutional Neural Networks with Application to Pulmonary Nodule Detection

Motivated by the problem of computer-aided detection (CAD) of pulmonary ...
research
02/20/2019

Emulating Human Developmental Stages with Bayesian Neural Networks

We compare the acquisition of knowledge in humans and machines. Research...
research
07/09/2013

Bayesian Discovery of Multiple Bayesian Networks via Transfer Learning

Bayesian network structure learning algorithms with limited data are bei...
research
02/06/2013

Cost-Sharing in Bayesian Knowledge Bases

Bayesian knowledge bases (BKBs) are a generalization of Bayes networks a...

Please sign up or login with your details

Forgot password? Click here to reset