Improving Neural Network Robustness through Neighborhood Preserving Layers

01/28/2021
by   Bingyuan Liu, et al.
0

Robustness against adversarial attack in neural networks is an important research topic in the machine learning community. We observe one major source of vulnerability of neural nets is from overparameterized fully-connected layers. In this paper, we propose a new neighborhood preserving layer which can replace these fully connected layers to improve the network robustness. We demonstrate a novel neural network architecture which can incorporate such layers and also can be trained efficiently. We theoretically prove that our models are more robust against distortion because they effectively control the magnitude of gradients. Finally, we empirically show that our designed network architecture is more robust against state-of-art gradient descent based attacks, such as a PGD attack on the benchmark datasets MNIST and CIFAR10.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/07/2019

FCC-GAN: A Fully Connected and Convolutional Net Architecture for GANs

Generative Adversarial Networks (GANs) are a powerful class of generativ...
research
03/10/2018

Attention-based Graph Neural Network for Semi-supervised Learning

Recently popularized graph neural networks achieve the state-of-the-art ...
research
07/06/2021

GradDiv: Adversarial Robustness of Randomized Neural Networks via Gradient Diversity Regularization

Deep learning is vulnerable to adversarial examples. Many defenses based...
research
06/20/2019

Clustering and Classification Networks

In this paper, we will describe a network architecture that demonstrates...
research
06/08/2021

Householder-Absolute Neural Layers For High Variability and Deep Trainability

We propose a new architecture for artificial neural networks called Hous...
research
11/09/2022

Designing Network Design Strategies Through Gradient Path Analysis

Designing a high-efficiency and high-quality expressive network architec...
research
06/22/2018

Fully Connected Networks and Generative Neural Networks Applied to Sclera Segmentation

Due to the world's demand for security systems, biometrics can be seen a...

Please sign up or login with your details

Forgot password? Click here to reset