Informative Gaussian Scale Mixture Priors for Bayesian Neural Networks

02/24/2020
by   Tianyu Cui, et al.
11

Encoding domain knowledge into the prior over the high-dimensional weight space is challenging in Bayesian neural networks. Two types of domain knowledge are commonly available in scientific applications: 1. feature sparsity (number of relevant features); 2. signal-to-noise ratio, quantified, for instance, as the proportion of variance explained (PVE). We show both types of domain knowledge can be encoded into the widely used Gaussian scale mixture priors with Automatic Relevance Determination. Specifically, we propose a new joint prior over the local (i.e., feature-specific) scale parameters to encode the knowledge about feature sparsity, and an algorithm to determine the global scale parameter (shared by all features) according to the PVE. Empirically, we show that the proposed informative prior improves prediction accuracy on publicly available datasets and in a genetics application.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/23/2020

Fine-Grain Few-Shot Vision via Domain Knowledge as Hyperspherical Priors

Prototypical networks have been shown to perform well at few-shot learni...
research
09/05/2017

Learning the PE Header, Malware Detection with Minimal Domain Knowledge

Many efforts have been made to use various forms of domain knowledge in ...
research
09/18/2018

Comparison between Suitable Priors for Additive Bayesian Networks

Additive Bayesian networks are types of graphical models that extend the...
research
02/27/2021

Incorporating Domain Knowledge into Deep Neural Networks

We present a survey of ways in which domain-knowledge has been included ...
research
11/16/2022

Learning unfolded networks with a cyclic group structure

Deep neural networks lack straightforward ways to incorporate domain kno...
research
03/08/2019

Consistent Bayesian Sparsity Selection for High-dimensional Gaussian DAG Models with Multiplicative and Beta-mixture Priors

Estimation of the covariance matrix for high-dimensional multivariate da...
research
10/26/2017

Laplacian Prior Variational Automatic Relevance Determination for Transmission Tomography

In the classic sparsity-driven problems, the fundamental L-1 penalty met...

Please sign up or login with your details

Forgot password? Click here to reset