Bayesian Neural Network via Stochastic Gradient Descent

05/29/2020
by   Abhinav Mishra, et al.
1

The goal of bayesian approach used in variational inference is to minimize the KL divergence between variational distribution and unknown posterior distribution. This is done by maximizing the Evidence Lower Bound (ELBO). A neural network is used to parametrize these distributions using Stochastic Gradient Descent. This work extends the work done by others by deriving the variational inference models. We show how SGD can be applied on bayesian neural networks by gradient estimation techniques. For validation, we have tested our model on 5 UCI datasets and the metrics chosen for evaluation are Root Mean Square Error (RMSE) error and negative log likelihood. Our work considerably beats the previous state of the art approaches for regression using bayesian neural networks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/27/2020

Stochastic Bayesian Neural Networks

Bayesian neural networks perform variational inference over weights but ...
research
04/19/2017

Importance Sampled Stochastic Optimization for Variational Inference

Variational inference approximates the posterior distribution of a proba...
research
04/06/2015

Early Stopping is Nonparametric Variational Inference

We show that unconverged stochastic gradient descent can be interpreted ...
research
02/09/2020

Projected Stein Variational Gradient Descent

The curse of dimensionality is a critical challenge in Bayesian inferenc...
research
12/22/2015

Implementing a Bayes Filter in a Neural Circuit: The Case of Unknown Stimulus Dynamics

In order to interact intelligently with objects in the world, animals mu...
research
11/16/2021

PredProp: Bidirectional Stochastic Optimization with Precision Weighted Predictive Coding

We present PredProp, a method for bidirectional, parallel and local opti...
research
06/19/2016

Tutorial on Variational Autoencoders

In just three years, Variational Autoencoders (VAEs) have emerged as one...

Please sign up or login with your details

Forgot password? Click here to reset