Lightweight Probabilistic Deep Networks

05/29/2018
by   Jochen Gast, et al.
6

Even though probabilistic treatments of neural networks have a long history, they have not found widespread use in practice. Sampling approaches are often too slow already for simple networks. The size of the inputs and the depth of typical CNN architectures in computer vision only compound this problem. Uncertainty in neural networks has thus been largely ignored in practice, despite the fact that it may provide important information about the reliability of predictions and the inner workings of the network. In this paper, we introduce two lightweight approaches to making supervised learning with probabilistic deep networks practical: First, we suggest probabilistic output layers for classification and regression that require only minimal changes to existing networks. Second, we employ assumed density filtering and show that activation uncertainties can be propagated in a practical fashion through the entire network, again with minor changes. Both probabilistic networks retain the predictive power of the deterministic counterpart, but yield uncertainties that correlate well with the empirical error induced by their predictions. Moreover, the robustness to adversarial examples is significantly increased.

READ FULL TEXT

page 6

page 14

page 15

page 16

page 17

page 18

research
01/15/2022

Robust uncertainty estimates with out-of-distribution pseudo-inputs training

Probabilistic models often use neural networks to control their predicti...
research
07/08/2017

Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks

Deep neural networks (DNNs) have excellent representative power and are ...
research
01/28/2019

Adversarial Examples Target Topological Holes in Deep Networks

It is currently unclear why adversarial examples are easy to construct f...
research
03/05/2019

Statistical Guarantees for the Robustness of Bayesian Neural Networks

We introduce a probabilistic robustness measure for Bayesian Neural Netw...
research
11/08/2017

Intriguing Properties of Adversarial Examples

It is becoming increasingly clear that many machine learning classifiers...
research
11/16/2019

Exploiting Oxide Based Resistive RAM Variability for Probabilistic AI Hardware Design

Uncertainty plays a key role in real-time machine learning. As a signifi...
research
09/12/2019

Generating Accurate Pseudo-labels via Hermite Polynomials for SSL Confidently

Rectified Linear Units (ReLUs) are among the most widely used activation...

Please sign up or login with your details

Forgot password? Click here to reset