Sharing Leaky-Integrate-and-Fire Neurons for Memory-Efficient Spiking Neural Networks

05/26/2023
by   Youngeun Kim, et al.
0

Spiking Neural Networks (SNNs) have gained increasing attention as energy-efficient neural networks owing to their binary and asynchronous computation. However, their non-linear activation, that is Leaky-Integrate-and-Fire (LIF) neuron, requires additional memory to store a membrane voltage to capture the temporal dynamics of spikes. Although the required memory cost for LIF neurons significantly increases as the input dimension goes larger, a technique to reduce memory for LIF neurons has not been explored so far. To address this, we propose a simple and effective solution, EfficientLIF-Net, which shares the LIF neurons across different layers and channels. Our EfficientLIF-Net achieves comparable accuracy with the standard SNNs while bringing up to  4.3X forward memory efficiency and  21.9X backward memory efficiency for LIF neurons. We conduct experiments on various datasets including CIFAR10, CIFAR100, TinyImageNet, ImageNet-100, and N-Caltech101. Furthermore, we show that our approach also offers advantages on Human Activity Recognition (HAR) datasets, which heavily rely on temporal information.

READ FULL TEXT
research
01/25/2022

Event-based Video Reconstruction via Potential-assisted Spiking Neural Network

Neuromorphic vision sensor is a new bio-inspired imaging paradigm that r...
research
11/12/2020

LIAF-Net: Leaky Integrate and Analog Fire Network for Lightweight and Efficient Spatiotemporal Information Processing

Spiking neural networks (SNNs) based on Leaky Integrate and Fire (LIF) m...
research
03/30/2022

SIT: A Bionic and Non-Linear Neuron for Spiking Neural Network

Spiking Neural Networks (SNNs) have piqued researchers' interest because...
research
01/23/2022

Neural Architecture Search for Spiking Neural Networks

Spiking Neural Networks (SNNs) have gained huge attention as a potential...
research
06/14/2021

Estimating the interaction graph of stochastic neuronal dynamics by observing only pairs of neurons

We address the questions of identifying pairs of interacting neurons fro...
research
08/02/2021

Formation of cell assemblies with iterative winners-take-all computation and excitation-inhibition balance

This paper targets the problem of encoding information into binary cell ...
research
03/23/2020

Inherent Adversarial Robustness of Deep Spiking Neural Networks: Effects of Discrete Input Encoding and Non-Linear Activations

In the recent quest for trustworthy neural networks, we present Spiking ...

Please sign up or login with your details

Forgot password? Click here to reset