Fundamental aspects of noise in analog-hardware neural networks

by   Nadezhda Semenova, et al.

We study and analyze the fundamental aspects of noise propagation in recurrent as well as deep, multi-layer networks. The main focus of our study are neural networks in analogue hardware, yet the methodology provides insight for networks in general. The system under study consists of noisy linear nodes, and we investigate the signal-to-noise ratio at the network's outputs which is the upper limit to such a system's computing accuracy. We consider additive and multiplicative noise which can be purely local as well as correlated across populations of neurons. This covers the chief internal-perturbations of hardware networks and noise amplitudes were obtained from a physically implemented recurrent neural network and therefore correspond to a real-world system. Analytic solutions agree exceptionally well with numerical data, enabling clear identification of the most critical components and aspects for noise management. Focusing on linear nodes isolates the impact of network connections and allows us to derive strategies for mitigating noise. Our work is the starting point in addressing this aspect of analogue neural networks, and our results identify notoriously sensitive points while simultaneously highlighting the robustness of such computational systems.


page 1

page 2

page 3

page 4


The general aspects of noise in analogue hardware deep neural networks

Deep neural networks unlocked a vast range of new applications by solvin...

Noise impact on recurrent neural network with linear activation function

In recent years, more and more researchers in the field of neural networ...

Boolean learning under noise-perturbations in hardware neural networks

A high efficiency hardware integration of neural networks benefits from ...

Walking Noise: Understanding Implications of Noisy Computations on Classification Tasks

Machine learning methods like neural networks are extremely successful a...

Noise mitigation strategies in physical feedforward neural networks

Physical neural networks are promising candidates for next generation ar...

Critical initialisation for deep signal propagation in noisy rectifier neural networks

Stochastic regularisation is an important weapon in the arsenal of a dee...

Robust Associative Memories Naturally Occuring From Recurrent Hebbian Networks Under Noise

The brain is a noisy system subject to energy constraints. These facts a...

Please sign up or login with your details

Forgot password? Click here to reset