TCJA-SNN: Temporal-Channel Joint Attention for Spiking Neural Networks

06/21/2022
by   Rui-Jie Zhu, et al.
0

Spiking Neural Networks (SNNs) is a practical approach toward more data-efficient deep learning by simulating neurons leverage on temporal information. In this paper, we propose the Temporal-Channel Joint Attention (TCJA) architectural unit, an efficient SNN technique that depends on attention mechanisms, by effectively enforcing the relevance of spike sequence along both spatial and temporal dimensions. Our essential technical contribution lies on: 1) compressing the spike stream into an average matrix by employing the squeeze operation, then using two local attention mechanisms with an efficient 1-D convolution to establish temporal-wise and channel-wise relations for feature extraction in a flexible fashion. 2) utilizing the Cross Convolutional Fusion (CCF) layer for modeling inter-dependencies between temporal and channel scope, which breaks the independence of the two dimensions and realizes the interaction between features. By virtue of jointly exploring and recalibrating data stream, our method outperforms the state-of-the-art (SOTA) by up to 15.7 in terms of top-1 classification accuracy on all tested mainstream static and neuromorphic datasets, including Fashion-MNIST, CIFAR10-DVS, N-Caltech 101, and DVS128 Gesture.

READ FULL TEXT

page 3

page 9

research
10/11/2022

STSC-SNN: Spatio-Temporal Synaptic Connection with Temporal Convolution and Attention for Spiking Neural Networks

Spiking Neural Networks (SNNs), as one of the algorithmic models in neur...
research
09/22/2022

A Spatial-channel-temporal-fused Attention for Spiking Neural Networks

Spiking neural networks (SNNs) mimic brain computational strategies, and...
research
04/22/2021

Noise-Robust Deep Spiking Neural Networks with Temporal Information

Spiking neural networks (SNNs) have emerged as energy-efficient neural n...
research
06/30/2021

Exploiting Spiking Dynamics with Spatial-temporal Feature Normalization in Graph Learning

Biological spiking neurons with intrinsic dynamics underlie the powerful...
research
02/19/2020

Exploiting Neuron and Synapse Filter Dynamics in Spatial Temporal Learning of Deep Spiking Neural Network

The recent discovered spatial-temporal information processing capability...
research
07/07/2020

Multivariate Time Series Classification Using Spiking Neural Networks

There is an increasing demand to process streams of temporal data in ene...
research
06/21/2023

Efficient Deep Spiking Multi-Layer Perceptrons with Multiplication-Free Inference

Advancements in adapting deep convolution architectures for Spiking Neur...

Please sign up or login with your details

Forgot password? Click here to reset