DeepAI AI Chat
Log In Sign Up

Robustness, Privacy, and Generalization of Adversarial Training

by   Fengxiang He, et al.
The University of Sydney

Adversarial training can considerably robustify deep neural networks to resist adversarial attacks. However, some works suggested that adversarial training might comprise the privacy-preserving and generalization abilities. This paper establishes and quantifies the privacy-robustness trade-off and generalization-robustness trade-off in adversarial training from both theoretical and empirical aspects. We first define a notion, robustified intensity to measure the robustness of an adversarial training algorithm. This measure can be approximate empirically by an asymptotically consistent empirical estimator, empirical robustified intensity. Based on the robustified intensity, we prove that (1) adversarial training is (ε, δ)-differentially private, where the magnitude of the differential privacy has a positive correlation with the robustified intensity; and (2) the generalization error of adversarial training can be upper bounded by an 𝒪(√(log N)/N) on-average bound and an 𝒪(1/√(N)) high-probability bound, both of which have positive correlations with the robustified intensity. Additionally, our generalization bounds do not explicitly rely on the parameter size which would be prohibitively large in deep learning. Systematic experiments on standard datasets, CIFAR-10 and CIFAR-100, are in full agreement with our theories. The source code package is available at <>.


page 1

page 2

page 3

page 4


Privacy Leakage of Adversarial Training Models in Federated Learning Systems

Adversarial Training (AT) is crucial for obtaining deep neural networks ...

Learning to be adversarially robust and differentially private

We study the difficulties in learning that arise from robust and differe...

Can collaborative learning be private, robust and scalable?

We investigate the effectiveness of combining differential privacy, mode...

Randomized Adversarial Training via Taylor Expansion

In recent years, there has been an explosion of research into developing...

Invertible Tabular GANs: Killing Two Birds with OneStone for Tabular Data Synthesis

Tabular data synthesis has received wide attention in the literature. Th...

Enhancing Adversarial Training with Second-Order Statistics of Weights

Adversarial training has been shown to be one of the most effective appr...

A Simple Structure For Building A Robust Model

As deep learning applications, especially programs of computer vision, a...