First-spike based visual categorization using reward-modulated STDP

05/25/2017
by   Milad Mozafari, et al.
0

Reinforcement learning (RL) has recently regained popularity, with major achievements such as beating the European game of Go champion. Here, for the first time, we show that RL can be used efficiently to train a spiking neural network (SNN) to perform object recognition in natural images without using an external classifier. We used a feedforward convolutional SNN and a temporal coding scheme where the most strongly activated neurons fire first, while less activated ones fire later, or not at all. In the highest layers, each neuron was assigned to an object category, and it was assumed that the stimulus category was the category of the first neuron to fire. If this assumption was correct, the neuron was rewarded, i.e. spike-timing-dependent plasticity (STDP) was applied, which reinforced the neuron's selectivity. Otherwise, anti-STDP was applied, which encouraged the neuron to learn something else. As demonstrated on various image datasets (Caltech, ETH-80, and NORB), this reward modulated STDP (R-STDP) approach extracted particularly discriminative visual features, whereas classic unsupervised STDP extracts any feature that consistently repeats. As a result, R-STDP outperformed STDP on these datasets. Furthermore, R-STDP is suitable for online learning, and can adapt to drastic changes such as label permutations. Finally, it is worth mentioning that both feature extraction and classification were done with spikes, using at most one spike per neuron. Thus the network is hardware friendly and energy efficient.

READ FULL TEXT

page 8

page 11

research
03/31/2018

Combining STDP and Reward-Modulated STDP in Deep Convolutional Spiking Neural Networks for Digit Recognition

The primate visual system has inspired the development of deep artificia...
research
11/04/2016

STDP-based spiking deep convolutional neural networks for object recognition

Previous studies have shown that spike-timing-dependent plasticity (STDP...
research
04/15/2015

Bio-inspired Unsupervised Learning of Visual Features Leads to Robust Invariant Object Recognition

Retinal image of surrounding objects varies tremendously due to the chan...
research
03/06/2019

SpykeTorch: Efficient Simulation of Convolutional Spiking Neural Networks with at most one Spike per Neuron

Application of deep convolutional spiking neural networks (SNNs) to arti...
research
04/09/2022

A Spiking Neural Network Structure Implementing Reinforcement Learning

At present, implementation of learning mechanisms in spiking neural netw...
research
06/03/2016

Acquisition of Visual Features Through Probabilistic Spike-Timing-Dependent Plasticity

The final version of this paper has been published in IEEEXplore availab...
research
04/21/2021

A Fully Spiking Hybrid Neural Network for Energy-Efficient Object Detection

This paper proposes a Fully Spiking Hybrid Neural Network (FSHNN) for en...

Please sign up or login with your details

Forgot password? Click here to reset