Robustness via curvature regularization, and vice versa

State-of-the-art classifiers have been shown to be largely vulnerable to adversarial perturbations. One of the most effective strategies to improve robustness is adversarial training. In this paper, we investigate the effect of adversarial training on the geometry of the classification landscape and decision boundaries. We show in particular that adversarial training leads to a significant decrease in the curvature of the loss surface with respect to inputs, leading to a drastically more "linear" behaviour of the network. Using a locally quadratic approximation, we provide theoretical evidence on the existence of a strong relation between large robustness and small curvature. To further show the importance of reduced curvature for improving the robustness, we propose a new regularizer that directly minimizes curvature of the loss surface, and leads to adversarial robustness that is on par with adversarial training. Besides being a more efficient and principled alternative to adversarial training, the proposed regularizer confirms our claims on the importance of exhibiting quasi-linear behavior in the vicinity of data points in order to achieve robustness.

READ FULL TEXT

page 1

page 3

page 4

page 5

page 6

page 7

page 9

page 11

research
12/09/2022

Understanding and Combating Robust Overfitting via Input Loss Landscape Analysis and Regularization

Adversarial training is widely used to improve the robustness of deep ne...
research
09/10/2020

Second Order Optimization for Adversarial Robustness and Interpretability

Deep neural networks are easily fooled by small perturbations known as a...
research
08/25/2021

Bridged Adversarial Training

Adversarial robustness is considered as a required property of deep neur...
research
07/04/2019

Adversarial Robustness through Local Linearization

Adversarial training is an effective methodology for training deep neura...
research
07/22/2019

Understanding Adversarial Robustness Through Loss Landscape Geometries

The pursuit of explaining and improving generalization in deep learning ...
research
08/26/2022

Lower Difficulty and Better Robustness: A Bregman Divergence Perspective for Adversarial Training

In this paper, we investigate on improving the adversarial robustness ob...
research
08/01/2023

Doubly Robust Instance-Reweighted Adversarial Training

Assigning importance weights to adversarial data has achieved great succ...

Please sign up or login with your details

Forgot password? Click here to reset