Towards Deep Learning Models Resistant to Large Perturbations

by   Amirreza Shaeiri, et al.

Adversarial robustness has proven to be a required property of machine learning algorithms. A key and often overlooked aspect of this problem is to try to make the adversarial noise magnitude as large as possible to enhance the benefits of the model robustness. We show that the well-established algorithm called "adversarial training" fails to train a deep neural network given a large, but reasonable, perturbation magnitude. In this paper, we propose a simple yet effective initialization of the network weights that makes learning on higher levels of noise possible. We next evaluate this idea rigorously on MNIST (ϵ up to ≈ 0.40) and CIFAR10 (ϵ up to ≈ 32/255) datasets assuming the ℓ_∞ attack model. Additionally, in order to establish the limits of ϵ in which the learning is feasible, we study the optimal robust classifier assuming full access to the joint data and label distribution. Then, we provide some theoretical results on the adversarial accuracy for a simple multi-dimensional Bernoulli distribution, which yields some insights on the range of feasible perturbations for the MNIST dataset.


page 7

page 29

page 30

page 31


A3T: Adversarially Augmented Adversarial Training

Recent research showed that deep neural networks are highly sensitive to...

Robust Deep Learning Models Against Semantic-Preserving Adversarial Attack

Deep learning models can be fooled by small l_p-norm adversarial perturb...

Lagrangian Objective Function Leads to Improved Unforeseen Attack Generalization in Adversarial Training

Recent improvements in deep learning models and their practical applicat...

Applying Tensor Decomposition to image for Robustness against Adversarial Attack

Nowadays the deep learning technology is growing faster and shows dramat...

Directional Adversarial Training for Cost Sensitive Deep Learning Classification Applications

In many real-world applications of Machine Learning it is of paramount i...

Constant Random Perturbations Provide Adversarial Robustness with Minimal Effect on Accuracy

This paper proposes an attack-independent (non-adversarial training) tec...

Model-Based Robust Deep Learning

While deep learning has resulted in major breakthroughs in many applicat...

Code Repositories


The codes necessary to reproduce the results of our preprint on how to do adversarial training on large perturbation norms.

view repo

Please sign up or login with your details

Forgot password? Click here to reset