Revisiting Loss Landscape for Adversarial Robustness

04/13/2020
by   Dongxian Wu, et al.
0

The study on improving the robustness of deep neural networks against adversarial examples grows rapidly in recent years. Among them, adversarial training is the most promising one, based on which, a lot of improvements have been developed, such as adding regularizations or leveraging unlabeled data. However, these improvements seem to come from isolated perspectives, so that we are curious about if there is something in common behind them. In this paper, we investigate the surface geometry of several well-recognized adversarial training variants, and reveal that their adversarial loss landscape is closely related to the adversarially robust generalization, i.e., the flatter the adversarial loss landscape, the smaller the adversarially robust generalization gap. Based on this finding, we then propose a simple yet effective module, Adversarial Weight Perturbation (AWP), to directly regularize the flatness of the adversarial loss landscape in the adversarial training framework. Extensive experiments demonstrate that AWP indeed owns flatter landscape and can be easily incorporated into various adversarial training variants to enhance their adversarial robustness further.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/30/2022

Robust Weight Perturbation for Adversarial Training

Overfitting widely exists in adversarial robust training of deep network...
research
03/19/2023

Randomized Adversarial Training via Taylor Expansion

In recent years, there has been an explosion of research into developing...
research
02/05/2021

Adversarial Training Makes Weight Loss Landscape Sharper in Logistic Regression

Adversarial training is actively studied for learning robust models agai...
research
04/09/2021

Relating Adversarially Robust Generalization to Flat Minima

Adversarial training (AT) has become the de-facto standard to obtain mod...
research
07/22/2019

Understanding Adversarial Robustness Through Loss Landscape Geometries

The pursuit of explaining and improving generalization in deep learning ...
research
05/09/2019

Exploring the Hyperparameter Landscape of Adversarial Robustness

Adversarial training shows promise as an approach for training models th...
research
08/25/2020

Likelihood Landscapes: A Unifying Principle Behind Many Adversarial Defenses

Convolutional Neural Networks have been shown to be vulnerable to advers...

Please sign up or login with your details

Forgot password? Click here to reset