Adversarial Spheres

01/09/2018
by   Justin Gilmer, et al.
0

State of the art computer vision models have been shown to be vulnerable to small adversarial perturbations of the input. In other words, most images in the data distribution are both correctly classified by the model and are very close to a visually similar misclassified image. Despite substantial research interest, the cause of the phenomenon is still poorly understood and remains unsolved. We hypothesize that this counter intuitive behavior is a naturally occurring result of the high dimensional geometry of the data manifold. As a first step towards exploring this hypothesis, we study a simple synthetic dataset of classifying between two concentric high dimensional spheres. For this dataset we show a fundamental tradeoff between the amount of test error and the average distance to nearest error. In particular, we prove that any model which misclassifies a small constant fraction of a sphere will be vulnerable to adversarial perturbations of size O(1/√(d)). Surprisingly, when we train several different architectures on this dataset, all of their error sets naturally approach this theoretical bound. As a result of the theory, the vulnerability of neural networks to small adversarial perturbations is a logical consequence of the amount of test error observed. We hope that our theoretical analysis of this very simple case will point the way forward to explore how the geometry of complex real-world data sets leads to adversarial examples.

READ FULL TEXT
research
02/23/2018

Adversarial vulnerability for any classifier

Despite achieving impressive and often superhuman performance on multipl...
research
06/28/2020

Geometry-Inspired Top-k Adversarial Perturbations

State-of-the-art deep learning models are untrustworthy due to their vul...
research
06/18/2021

The Dimpled Manifold Model of Adversarial Examples in Machine Learning

The extreme fragility of deep neural networks when presented with tiny p...
research
06/06/2020

Unique properties of adversarially trained linear classifiers on Gaussian data

Machine learning models are vulnerable to adversarial perturbations, tha...
research
12/04/2018

Adversarial Example Decomposition

Research has shown that widely used deep neural networks are vulnerable ...
research
02/22/2019

On the Sensitivity of Adversarial Robustness to Input Data Distributions

Neural networks are vulnerable to small adversarial perturbations. Exist...
research
06/08/2019

Sensitivity of Deep Convolutional Networks to Gabor Noise

Deep Convolutional Networks (DCNs) have been shown to be sensitive to Un...

Please sign up or login with your details

Forgot password? Click here to reset