Learning from Event Cameras with Sparse Spiking Convolutional Neural Networks

by   Loïc Cordone, et al.

Convolutional neural networks (CNNs) are now the de facto solution for computer vision problems thanks to their impressive results and ease of learning. These networks are composed of layers of connected units called artificial neurons, loosely modeling the neurons in a biological brain. However, their implementation on conventional hardware (CPU/GPU) results in high power consumption, making their integration on embedded systems difficult. In a car for example, embedded algorithms have very high constraints in term of energy, latency and accuracy. To design more efficient computer vision algorithms, we propose to follow an end-to-end biologically inspired approach using event cameras and spiking neural networks (SNNs). Event cameras output asynchronous and sparse events, providing an incredibly efficient data source, but processing these events with synchronous and dense algorithms such as CNNs does not yield any significant benefits. To address this limitation, we use spiking neural networks (SNNs), which are more biologically realistic neural networks where units communicate using discrete spikes. Due to the nature of their operations, they are hardware friendly and energy-efficient, but training them still remains a challenge. Our method enables the training of sparse spiking convolutional neural networks directly on event data, using the popular deep learning framework PyTorch. The performances in terms of accuracy, sparsity and training time on the popular DVS128 Gesture Dataset make it possible to use this bio-inspired approach for the future embedding of real-time applications on low-power neuromorphic hardware.


page 1

page 4


Object Detection with Spiking Neural Networks on Automotive Event Data

Automotive embedded algorithms have very high constraints in terms of la...

Closing the Accuracy Gap in an Event-Based Visual Recognition Task

Mobile and embedded applications require neural networks-based pattern r...

Deep counter networks for asynchronous event-based processing

Despite their advantages in terms of computational resources, latency, a...

Diagnostic data integration using deep neural networks for real-time plasma analysis

Recent advances in acquisition equipment is providing experiments with g...

Effective AER Object Classification Using Segmented Probability-Maximization Learning in Spiking Neural Networks

Address event representation (AER) cameras have recently attracted more ...

Spiking Approximations of the MaxPooling Operation in Deep SNNs

Spiking Neural Networks (SNNs) are an emerging domain of biologically in...

A New Neuromorphic Computing Approach for Epileptic Seizure Prediction

Several high specificity and sensitivity seizure prediction methods with...