Exploring the Connection Between Binary and Spiking Neural Networks

02/24/2020
by   Sen Lu, et al.
0

On-chip edge intelligence has necessitated the exploration of algorithmic techniques to reduce the compute requirements of current machine learning frameworks. This work aims to bridge the recent algorithmic progress in training Binary Neural Networks and Spiking Neural Networks - both of which are driven by the same motivation and yet synergies between the two have not been fully explored. We show that training Spiking Neural Networks in the extreme quantization regime results in near full precision accuracies on large-scale datasets like CIFAR-100 and ImageNet. An important implication of this work is that Binary Spiking Neural Networks can be enabled by "In-Memory" hardware accelerators catered for Binary Neural Networks without suffering any accuracy degradation due to binarization. We utilize standard training techniques for non-spiking networks to generate our spiking networks by conversion process and also perform an extensive empirical analysis and explore simple design-time and run-time optimization techniques for reducing inference latency of spiking networks (both for binary and full-precision models) by an order of magnitude over prior work.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/04/2022

Optimizing the Consumption of Spiking Neural Networks with Activity Regularization

Reducing energy consumption is a critical point for neural network model...
research
05/30/2023

Low Precision Quantization-aware Training in Spiking Neural Networks with Differentiable Quantization Function

Deep neural networks have been proven to be highly effective tools in va...
research
06/21/2022

Structural Stability of Spiking Neural Networks

The past decades have witnessed an increasing interest in spiking neural...
research
11/22/2022

Fast Exploration of the Impact of Precision Reduction on Spiking Neural Networks

Approximate Computing (AxC) techniques trade off the computation accurac...
research
12/19/2019

Spiking Networks for Improved Cognitive Abilities of Edge Computing Devices

This concept paper highlights a recently opened opportunity for large sc...
research
01/18/2022

FPGA-optimized Hardware acceleration for Spiking Neural Networks

Artificial intelligence (AI) is gaining success and importance in many d...
research
04/29/2019

Neuromorphic Acceleration for Approximate Bayesian Inference on Neural Networks via Permanent Dropout

As neural networks have begun performing increasingly critical tasks for...

Please sign up or login with your details

Forgot password? Click here to reset