DeepAI AI Chat
Log In Sign Up

StereoSpike: Depth Learning with a Spiking Neural Network

by   Ulysse Rançon, et al.
Université de Bordeaux

Depth estimation is an important computer vision task, useful in particular for navigation in autonomous vehicles, or for object manipulation in robotics. Here we solved it using an end-to-end neuromorphic approach, combining two event-based cameras and a Spiking Neural Network (SNN) with a slightly modified U-Net-like encoder-decoder architecture, that we named StereoSpike. More specifically, we used the Multi Vehicle Stereo Event Camera Dataset (MVSEC). It provides a depth ground-truth, which was used to train StereoSpike in a supervised manner, using surrogate gradient descent. We propose a novel readout paradigm to obtain a dense analog prediction – the depth of each pixel – from the spikes of the decoder. We demonstrate that this architecture generalizes very well, even better than its non-spiking counterparts, leading to state-of-the-art test accuracy. To the best of our knowledge, it is the first time that such a large-scale regression problem is solved by a fully spiking network. Finally, we show that low firing rates (<10 regularization, with a minimal cost in accuracy. This means that StereoSpike could be efficiently implemented on neuromorphic chips, opening the door for low power and real time embedded systems.


MSS-DepthNet: Depth Prediction with Multi-Step Spiking Neural Network

Event cameras are considered to have great potential for computer vision...

Improving Surrogate Gradient Learning in Spiking Neural Networks via Regularization and Normalization

Spiking neural networks (SNNs) are different from the classical networks...

Optical Flow estimation with Event-based Cameras and Spiking Neural Networks

Event-based cameras are raising interest within the computer vision comm...

ES-ImageNet: A Million Event-Stream Classification Dataset for Spiking Neural Networks

With event-driven algorithms, especially the spiking neural networks (SN...

LaneSNNs: Spiking Neural Networks for Lane Detection on the Loihi Neuromorphic Processor

Autonomous Driving (AD) related features represent important elements fo...

Neuro-Planner: A 3D Visual Navigation Method for MAV with Depth Camera based on Neuromorphic Reinforcement Learning

Traditional visual navigation methods of micro aerial vehicle (MAV) usua...