Strategy and Benchmark for Converting Deep Q-Networks to Event-Driven Spiking Neural Networks

by   Weihao Tan, et al.

Spiking neural networks (SNNs) have great potential for energy-efficient implementation of Deep Neural Networks (DNNs) on dedicated neuromorphic hardware. Recent studies demonstrated competitive performance of SNNs compared with DNNs on image classification tasks, including CIFAR-10 and ImageNet data. The present work focuses on using SNNs in combination with deep reinforcement learning in ATARI games, which involves additional complexity as compared to image classification. We review the theory of converting DNNs to SNNs and extending the conversion to Deep Q-Networks (DQNs). We propose a robust representation of the firing rate to reduce the error during the conversion process. In addition, we introduce a new metric to evaluate the conversion process by comparing the decisions made by the DQN and SNN, respectively. We also analyze how the simulation time and parameter normalization influence the performance of converted SNNs. We achieve competitive scores on 17 top-performing Atari games. To the best of our knowledge, our work is the first to achieve state-of-the-art performance on multiple Atari games with SNNs. Our work serves as a benchmark for the conversion of DQNs to SNNs and paves the way for further research on solving reinforcement learning tasks with SNNs.


page 1

page 2

page 3

page 4


Human-Level Control through Directly-Trained Deep Spiking Q-Networks

As the third-generation neural networks, Spiking Neural Networks (SNNs) ...

Designing Interpretable Approximations to Deep Reinforcement Learning with Soft Decision Trees

In an ever expanding set of research and application areas, deep neural ...

Sneaky Spikes: Uncovering Stealthy Backdoor Attacks in Spiking Neural Networks with Neuromorphic Data

Deep neural networks (DNNs) have achieved excellent results in various t...

Improved robustness of reinforcement learning policies upon conversion to spiking neuronal network platforms applied to ATARI games

Various implementations of Deep Reinforcement Learning (RL) demonstrated...

Theory and Tools for the Conversion of Analog to Spiking Convolutional Neural Networks

Deep convolutional neural networks (CNNs) have shown great potential for...

One Timestep is All You Need: Training Spiking Neural Networks with Ultra Low Latency

Spiking Neural Networks (SNNs) are energy efficient alternatives to comm...

Reinforcement Learning with Low-Complexity Liquid State Machines

We propose reinforcement learning on simple networks consisting of rando...

Please sign up or login with your details

Forgot password? Click here to reset