Noise Optimization for Artificial Neural Networks

02/06/2021
by   Li Xiao, et al.
0

Adding noises to artificial neural network(ANN) has been shown to be able to improve robustness in previous work. In this work, we propose a new technique to compute the pathwise stochastic gradient estimate with respect to the standard deviation of the Gaussian noise added to each neuron of the ANN. By our proposed technique, the gradient estimate with respect to noise levels is a byproduct of the backpropagation algorithm for estimating gradient with respect to synaptic weights in ANN. Thus, the noise level for each neuron can be optimized simultaneously in the processing of training the synaptic weights at nearly no extra computational cost. In numerical experiments, our proposed method can achieve significant performance improvement on robustness of several popular ANN structures under both black box and white box attacks tested in various computer vision datasets.

READ FULL TEXT
research
07/12/2017

The detector principle of constructing artificial neural networks as an alternative to the connectionist paradigm

Artificial neural networks (ANN) are inadequate to biological neural net...
research
12/21/2014

SENNS: Sparse Extraction Neural NetworkS for Feature Extraction

By drawing on ideas from optimisation theory, artificial neural networks...
research
03/04/2020

Colored Noise Injection for Training Adversarially Robust Neural Networks

Even though deep learning have shown unmatched performance on various ta...
research
11/25/2014

Hypotheses of neural code and the information model of the neuron-detector

This paper deals with the problem of neural code solving. On the basis o...
research
05/25/2023

Dendritic Integration Based Quadratic Neural Networks Outperform Traditional Aritificial Ones

Incorporating biological neuronal properties into Artificial Neural Netw...
research
05/20/2019

The Twin-System Approach as One Generic Solution for XAI: An Overview of ANN-CBR Twins for Explaining Deep Learning

The notion of twin systems is proposed to address the eXplainable AI (XA...
research
03/23/2023

Noise impact on recurrent neural network with linear activation function

In recent years, more and more researchers in the field of neural networ...

Please sign up or login with your details

Forgot password? Click here to reset