Adaptive versus Standard Descent Methods and Robustness Against Adversarial Examples

11/09/2019
by   Marc Khoury, et al.
12

Adversarial examples are a pervasive phenomenon of machine learning models where seemingly imperceptible perturbations to the input lead to misclassifications for otherwise statistically accurate models. In this paper we study how the choice of optimization algorithm influences the robustness of the resulting classifier to adversarial examples. Specifically we show an example of a learning problem for which the solution found by adaptive optimization algorithms exhibits qualitatively worse robustness properties against both L_2- and L_∞-adversaries than the solution found by non-adaptive algorithms. Then we fully characterize the geometry of the loss landscape of L_2-adversarial training in least-squares linear regression. The geometry of the loss landscape is subtle and has important consequences for optimization algorithms. Finally we provide experimental evidence which suggests that non-adaptive methods consistently produce more robust models than adaptive methods.

READ FULL TEXT

page 8

page 12

page 13

research
11/01/2018

On the Geometry of Adversarial Examples

Adversarial examples are a pervasive phenomenon of machine learning mode...
research
10/14/2019

Confidence-Calibrated Adversarial Training: Towards Robust Models Generalizing Beyond the Attack Used During Training

Adversarial training is the standard to train models robust against adve...
research
02/05/2021

Adversarial Training Makes Weight Loss Landscape Sharper in Logistic Regression

Adversarial training is actively studied for learning robust models agai...
research
10/02/2022

Adaptive Smoothness-weighted Adversarial Training for Multiple Perturbations with Its Stability Analysis

Adversarial Training (AT) has been demonstrated as one of the most effec...
research
05/02/2019

Adversarial Training with Voronoi Constraints

Adversarial examples are a pervasive phenomenon of machine learning mode...
research
05/28/2021

Towards optimally abstaining from prediction

A common challenge across all areas of machine learning is that training...
research
04/09/2021

Relating Adversarially Robust Generalization to Flat Minima

Adversarial training (AT) has become the de-facto standard to obtain mod...

Please sign up or login with your details

Forgot password? Click here to reset