Asymptotic Behavior of Adversarial Training in Binary Classification

10/26/2020
by   Hossein Taheri, et al.
0

It is widely known that several machine learning models are susceptible to adversarial attacks i.e., small adversarial perturbations applied to data points causing the model to misclassify the data. Adversarial training using empirical risk minimization methods, is the state-of-the-art method for defense against adversarial attacks. Despite being successful, several problems in understanding generalization performance of adversarial training remain open. In this paper, we derive precise theoretical predictions for the performance of adversarial training in binary linear classification. We consider the modern high-dimensional regime where the dimension of data grows with the size of the training dataset at a constant ratio. Our results provide exact asymptotics for the performance of estimators obtained by adversarial training with ℓ_q-norm bounded perturbations (q ≥ 1) and for binary labels and Gaussian features. These sharp predictions enable us to explore the role of various factors including over-parametrization ratio, data model and attack budget on the performance of adversarial training.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/09/2021

Attacking Adversarial Attacks as A Defense

It is well known that adversarial attacks can fool deep neural networks ...
research
06/07/2018

On Adversarial Risk and Training

In this work we formally define the notions of adversarial perturbations...
research
06/21/2023

Adversarial Training with Generated Data in High-Dimensional Regression: An Asymptotic Study

In recent years, studies such as <cit.> have demonstrated that incorpora...
research
11/28/2022

Gamma-convergence of a nonlocal perimeter arising in adversarial machine learning

In this paper we prove Gamma-convergence of a nonlocal perimeter of Mink...
research
10/21/2020

Precise Statistical Analysis of Classification Accuracies for Adversarial Training

Despite the wide empirical success of modern machine learning algorithms...
research
11/26/2021

The Geometry of Adversarial Training in Binary Classification

We establish an equivalence between a family of adversarial training pro...
research
06/18/2022

The Consistency of Adversarial Training for Binary Classification

Robustness to adversarial perturbations is of paramount concern in moder...

Please sign up or login with your details

Forgot password? Click here to reset