Neuromorphic Acceleration for Approximate Bayesian Inference on Neural Networks via Permanent Dropout

by   Nathan Wycoff, et al.

As neural networks have begun performing increasingly critical tasks for society, ranging from driving cars to identifying candidates for drug development, the value of their ability to perform uncertainty quantification (UQ) in their predictions has risen commensurately. Permanent dropout, a popular method for neural network UQ, involves injecting stochasticity into the inference phase of the model and creating many predictions for each of the test data. This shifts the computational and energy burden of deep neural networks from the training phase to the inference phase. Recent work has demonstrated near-lossless conversion of classical deep neural networks to their spiking counterparts. We use these results to demonstrate the feasibility of conducting the inference phase with permanent dropout on spiking neural networks, mitigating the technique's computational and energy burden, which is essential for its use at scale or on edge platforms. We demonstrate the proposed approach via the Nengo spiking neural simulator on a combination drug therapy dataset for cancer treatment, where UQ is critical. Our results indicate that the spiking approximation gives a predictive distribution practically indistinguishable from that given by the classical network.


Deep Spiking Convolutional Neural Network for Single Object Localization Based On Deep Continuous Local Learning

With the advent of neuromorphic hardware, spiking neural networks can be...

Optimizing the Consumption of Spiking Neural Networks with Activity Regularization

Reducing energy consumption is a critical point for neural network model...

Exploring the Connection Between Binary and Spiking Neural Networks

On-chip edge intelligence has necessitated the exploration of algorithmi...

Efficient Uncertainty Estimation in Spiking Neural Networks via MC-dropout

Spiking neural networks (SNNs) have gained attention as models of sparse...

Temporal Knowledge Sharing enable Spiking Neural Network Learning from Past and Future

Spiking neural networks have attracted extensive attention from research...

Membrane Potential Batch Normalization for Spiking Neural Networks

As one of the energy-efficient alternatives of conventional neural netwo...

Please sign up or login with your details

Forgot password? Click here to reset