On the Effects of Quantisation on Model Uncertainty in Bayesian Neural Networks

02/22/2021
by   Martin Ferianc, et al.
11

Bayesian neural networks (BNNs) are making significant progress in many research areas where decision making needs to be accompanied by uncertainty estimation. Being able to quantify uncertainty while making decisions is essential for understanding when the model is over-/under-confident, and hence BNNs are attracting interest in safety-critical applications, such as autonomous driving, healthcare and robotics. Nevertheless, BNNs have not been as widely used in industrial practice, mainly because of their increased memory and compute costs. In this work, we investigate quantisation of BNNs by compressing 32-bit floating-point weights and activations to their integer counterparts, that has already been successful in reducing the compute demand in standard pointwise neural networks. We study three types of quantised BNNs, we evaluate them under a wide range of different settings, and we empirically demonstrate that an uniform quantisation scheme applied to BNNs does not substantially decrease their quality of uncertainty estimation.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 5

page 8

page 13

05/12/2021

High-Performance FPGA-based Accelerator for Bayesian Neural Networks

Neural networks (NNs) have demonstrated their potential in a wide range ...
11/30/2018

Ensemble Bayesian Decision Making with Redundant Deep Perceptual Control Policies

This work presents a novel ensemble of Bayesian Neural Networks (BNNs) f...
09/21/2019

Uncertainty Quantification with Statistical Guarantees in End-to-End Autonomous Driving Control

Deep neural network controllers for autonomous driving have recently ben...
07/07/2021

A Survey of Uncertainty in Deep Neural Networks

Due to their increasing spread, confidence in neural network predictions...
05/21/2021

Ensemble Quantile Networks: Uncertainty-Aware Reinforcement Learning with Applications in Autonomous Driving

Reinforcement learning (RL) can be used to create a decision-making agen...
05/01/2018

Computable decision making on the reals and other spaces via partiality and nondeterminism

Though many safety-critical software systems use floating point to repre...
08/16/2021

Towards a Safety Case for Hardware Fault Tolerance in Convolutional Neural Networks Using Activation Range Supervision

Convolutional neural networks (CNNs) have become an established part of ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.