DeepAI
Log In Sign Up

Provable Robustness of Adversarial Training for Learning Halfspaces with Noise

04/19/2021
by   Difan Zou, et al.
8

We analyze the properties of adversarial training for learning adversarially robust halfspaces in the presence of agnostic label noise. Denoting 𝖮𝖯𝖳_p,r as the best robust classification error achieved by a halfspace that is robust to perturbations of ℓ_p balls of radius r, we show that adversarial training on the standard binary cross-entropy loss yields adversarially robust halfspaces up to (robust) classification error Õ(√(𝖮𝖯𝖳_2,r)) for p=2, and Õ(d^1/4√(𝖮𝖯𝖳_∞, r) + d^1/2𝖮𝖯𝖳_∞,r) when p=∞. Our results hold for distributions satisfying anti-concentration properties enjoyed by log-concave isotropic distributions among others. We additionally show that if one instead uses a nonconvex sigmoidal loss, adversarial training yields halfspaces with an improved robust classification error of O(𝖮𝖯𝖳_2,r) for p=2, and O(d^1/4𝖮𝖯𝖳_∞, r) when p=∞. To the best of our knowledge, this is the first work to show that adversarial training provably yields robust classifiers in the presence of noise.

READ FULL TEXT

page 1

page 2

page 3

page 4

11/19/2021

Fooling Adversarial Training with Inducing Noise

Adversarial training is widely believed to be a reliable approach to imp...
06/07/2018

On Adversarial Risk and Training

In this work we formally define the notions of adversarial perturbations...
10/07/2021

Double Descent in Adversarial Training: An Implicit Label Noise Perspective

Here, we show that the robust overfitting shall be viewed as the early p...
01/08/2020

MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius

Adversarial training is one of the most popular ways to learn robust mod...
06/18/2022

The Consistency of Adversarial Training for Binary Classification

Robustness to adversarial perturbations is of paramount concern in moder...
02/16/2020

Over-parameterized Adversarial Training: An Analysis Overcoming the Curse of Dimensionality

Adversarial training is a popular method to give neural nets robustness ...
02/10/2021

Bayesian Inference with Certifiable Adversarial Robustness

We consider adversarial training of deep neural networks through the len...