Bayesian Inference on Binary Spiking Networks Leveraging Nanoscale Device Stochasticity

02/02/2023
by   Prabodh Katti, et al.
0

Bayesian Neural Networks (BNNs) can overcome the problem of overconfidence that plagues traditional frequentist deep neural networks, and are hence considered to be a key enabler for reliable AI systems. However, conventional hardware realizations of BNNs are resource intensive, requiring the implementation of random number generators for synaptic sampling. Owing to their inherent stochasticity during programming and read operations, nanoscale memristive devices can be directly leveraged for sampling, without the need for additional hardware resources. In this paper, we introduce a novel Phase Change Memory (PCM)-based hardware implementation for BNNs with binary synapses. The proposed architecture consists of separate weight and noise planes, in which PCM cells are configured and operated to represent the nominal values of weights and to generate the required noise for sampling, respectively. Using experimentally observed PCM noise characteristics, for the exemplary Breast Cancer Dataset classification problem, we obtain hardware accuracy and expected calibration error matching that of an 8-bit fixed-point (FxP8) implementation, with projected savings of over 9× in terms of core area transistor count.

READ FULL TEXT

page 1

page 4

research
12/29/2019

MTJ-Based Hardware Synapse Design for Quantized Deep Neural Networks

Quantized neural networks (QNNs) are being actively researched as a solu...
research
06/06/2019

Stochasticity and Robustness in Spiking Neural Networks

Artificial neural networks normally require precise weights to operate, ...
research
03/25/2021

Enabling Incremental Training with Forward Pass for Edge Devices

Deep Neural Networks (DNNs) are commonly deployed on end devices that ex...
research
02/21/2022

Variation Aware Training of Hybrid Precision Neural Networks with 28nm HKMG FeFET Based Synaptic Core

This work proposes a hybrid-precision neural network training framework ...
research
09/28/2018

Learning Recurrent Binary/Ternary Weights

Recurrent neural networks (RNNs) have shown excellent performance in pro...
research
02/20/2021

Neural Sampling Machine with Stochastic Synapse allows Brain-like Learning and Inference

Many real-world mission-critical applications require continual online l...
research
11/07/2016

Fixed-point Factorized Networks

In recent years, Deep Neural Networks (DNN) based methods have achieved ...

Please sign up or login with your details

Forgot password? Click here to reset