Bayesian Continual Learning via Spiking Neural Networks

08/29/2022
by   Nicolas Skatchkovsky, et al.
32

Among the main features of biological intelligence are energy efficiency, capacity for continual adaptation, and risk management via uncertainty quantification. Neuromorphic engineering has been thus far mostly driven by the goal of implementing energy-efficient machines that take inspiration from the time-based computing paradigm of biological brains. In this paper, we take steps towards the design of neuromorphic systems that are capable of adaptation to changing learning tasks, while producing well-calibrated uncertainty quantification estimates. To this end, we derive online learning rules for spiking neural networks (SNNs) within a Bayesian continual learning framework. In it, each synaptic weight is represented by parameters that quantify the current epistemic uncertainty resulting from prior knowledge and observed data. The proposed online rules update the distribution parameters in a streaming fashion as data are observed. We instantiate the proposed approach for both real-valued and binary synaptic weights. Experimental results using Intel's Lava platform show the merits of Bayesian over frequentist learning in terms of capacity for adaptation and uncertainty quantification.

READ FULL TEXT

page 13

page 14

page 15

page 16

research
06/27/2022

On-device Synaptic Memory Consolidation using Fowler-Nordheim Quantum-tunneling

Synaptic memory consolidation has been heralded as one of the key mechan...
research
05/23/2022

Memory-enriched computation and learning in spiking neural networks through Hebbian plasticity

Memory is a key component of biological neural systems that enables the ...
research
08/09/2023

Enhancing Efficient Continual Learning with Dynamic Structure Development of Spiking Neural Networks

Children possess the ability to learn multiple cognitive tasks sequentia...
research
09/18/2023

Adaptive Reorganization of Neural Pathways for Continual Learning with Hybrid Spiking Neural Networks

The human brain can self-organize rich and diverse sparse neural pathway...
research
06/02/2023

Synaptic motor adaptation: A three-factor learning rule for adaptive robotic control in spiking neural networks

Legged robots operating in real-world environments must possess the abil...
research
07/21/2023

Neuromorphic Online Learning for Spatiotemporal Patterns with a Forward-only Timeline

Spiking neural networks (SNNs) are bio-plausible computing models with h...
research
10/21/2018

Training Dynamic Exponential Family Models with Causal and Lateral Dependencies for Generalized Neuromorphic Computing

Neuromorphic hardware platforms, such as Intel's Loihi chip, support the...

Please sign up or login with your details

Forgot password? Click here to reset