R3Net: Random Weights, Rectifier Linear Units and Robustness for Artificial Neural Network

03/12/2018
by   Arun Venkitaraman, et al.
0

We consider a neural network architecture with randomized features, a sign-splitter, followed by rectified linear units (ReLU). We prove that our architecture exhibits robustness to the input perturbation: the output feature of the neural network exhibits a Lipschitz continuity in terms of the input perturbation. We further show that the network output exhibits a discrimination ability that inputs that are not arbitrarily close generate output vectors which maintain distance between each other obeying a certain lower bound. This ensures that two different inputs remain discriminable while contracting the distance in the output feature space.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/30/2020

Locally Linear Attributes of ReLU Neural Networks

A ReLU neural network determines/is a continuous piecewise linear map fr...
research
10/02/2019

Identifying Weights and Architectures of Unknown ReLU Networks

The output of a neural network depends on its parameters in a highly non...
research
02/09/2020

Input Validation for Neural Networks via Runtime Local Robustness Verification

Local robustness verification can verify that a neural network is robust...
research
01/30/2018

Robustness of classification ability of spiking neural networks

It is well-known that the robustness of artificial neural networks (ANNs...
research
01/03/2023

Improving Performance in Neural Networks by Dendrites-Activated Connections

Computational units in artificial neural networks compute a linear combi...
research
02/28/2017

Deep Semi-Random Features for Nonlinear Function Approximation

We propose semi-random features for nonlinear function approximation. Th...
research
04/25/2020

NullSpaceNet: Nullspace Convoluional Neural Network with Differentiable Loss Function

We propose NullSpaceNet, a novel network that maps from the pixel level ...

Please sign up or login with your details

Forgot password? Click here to reset