SiamSNN: Spike-based Siamese Network for Energy-Efficient and Real-time Object Tracking

by   Yihao Luo, et al.

Although deep neural networks (DNNs) have achieved fantastic success in various scenarios, it's difficult to employ DNNs on many systems with limited resources due to their high energy consumption. It's well known that spiking neural networks (SNNs) are attracting more attention due to the capability of energy-efficient computing. Recently many works focus on converting DNNs into SNNs with little accuracy degradation in image classification on MNIST, CIFAR-10/100. However, few studies on shortening latency, and spike-based modules of more challenging tasks on complex datasets. In this paper, we focus on the similarity matching method of deep spike features and present a first spike-based Siamese network for object tracking called SiamSNN. Specifically, we propose a hybrid spiking similarity matching method with membrane potential and time step to evaluate the response map between exemplar and candidate images, with the same function as correlation layer in SiamFC. Then we present a coding scheme for utilizing temporal information of spike trains, and implement it in output spiking layers to improve the performance and shorten the latency. Our experiments show that SiamSNN achieves short latency and low precision loss of the original SiamFC on the tracking datasets OTB-2013, OTB-2015 and VOT2016. Moreover, SiamSNN achieves real-time (50 FPS) and extremely low energy consumption on TrueNorth.


page 1

page 7


Training Energy-Efficient Deep Spiking Neural Networks with Time-to-First-Spike Coding

The tremendous energy consumption of deep neural networks (DNNs) has bec...

Supervised Training of Siamese Spiking Neural Networks with Earth's Mover Distance

This study adapts the highly-versatile siamese neural network model to t...

Spiking-GAN: A Spiking Generative Adversarial Network Using Time-To-First-Spike Coding

Spiking Neural Networks (SNNs) have shown great potential in solving dee...

Spiking-YOLO: Spiking Neural Network for Real-time Object Detection

Over the past decade, deep neural networks (DNNs) have become a de-facto...

Deep SCNN-based Real-time Object Detection for Self-driving Vehicles Using LiDAR Temporal Data

Real-time accurate detection of three-dimensional (3D) objects is a fund...

Spiking NeRF: Making Bio-inspired Neural Networks See through the Real World

Spiking neuron networks (SNNs) have been thriving on numerous tasks to l...

Optimisation of a Siamese Neural Network for Real-Time Energy Efficient Object Tracking

In this paper the research on optimisation of visual object tracking usi...

Please sign up or login with your details

Forgot password? Click here to reset