How benign is benign overfitting?

07/08/2020
by   Amartya Sanyal, et al.
7

We investigate two causes for adversarial vulnerability in deep neural networks: bad data and (poorly) trained models. When trained with SGD, deep neural networks essentially achieve zero training error, even in the presence of label noise, while also exhibiting good generalization on natural test data, something referred to as benign overfitting [2, 10]. However, these models are vulnerable to adversarial attacks. We identify label noise as one of the causes for adversarial vulnerability, and provide theoretical and empirical evidence in support of this. Surprisingly, we find several instances of label noise in datasets such as MNIST and CIFAR, and that robustly trained models incur training error on some of these, i.e. they don't fit the noise. However, removing noisy labels alone does not suffice to achieve adversarial robustness. Standard training procedures bias neural networks towards learning "simple" classification boundaries, which may be less robust than more complex ones. We observe that adversarial training does produce more complex decision boundaries. We conjecture that in part the need for complex decision boundaries arises from sub-optimal representation learning. By means of simple toy examples, we show theoretically how the choice of representation can drastically affect adversarial robustness.

READ FULL TEXT

page 2

page 9

page 11

page 12

research
05/13/2019

Harnessing the Vulnerability of Latent Layers in Adversarially Trained Models

Neural networks are vulnerable to adversarial attacks -- small visually ...
research
03/14/2020

VarMixup: Exploiting the Latent Space for Robust Training and Inference

The vulnerability of Deep Neural Networks (DNNs) to adversarial attacks ...
research
03/03/2023

Certified Robust Neural Networks: Generalization and Corruption Resistance

Adversarial training aims to reduce the problematic susceptibility of mo...
research
05/25/2018

Topological Data Analysis of Decision Boundaries with Application to Model Selection

We propose the labeled Čech complex, the plain labeled Vietoris-Rips com...
research
07/08/2022

A law of adversarial risk, interpolation, and label noise

In supervised learning, it has been shown that label noise in the data c...
research
03/01/2021

Explaining Adversarial Vulnerability with a Data Sparsity Hypothesis

Despite many proposed algorithms to provide robustness to deep learning ...
research
03/07/2020

Geometry and Topology of Deep Neural Networks' Decision Boundaries

Geometry and topology of decision regions are closely related with class...

Please sign up or login with your details

Forgot password? Click here to reset