Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustness

10/15/2020
by   Long Zhao, et al.
7

Adversarial data augmentation has shown promise for training robust deep neural networks against unforeseen data shifts or corruptions. However, it is difficult to define heuristics to generate effective fictitious target distributions containing "hard" adversarial perturbations that are largely different from the source distribution. In this paper, we propose a novel and effective regularization term for adversarial data augmentation. We theoretically derive it from the information bottleneck principle, which results in a maximum-entropy formulation. Intuitively, this regularization term encourages perturbing the underlying source distribution to enlarge predictive uncertainty of the current model, so that the generated "hard" adversarial perturbations can improve the model robustness during training. Experimental results on three standard benchmarks demonstrate that our method consistently outperforms the existing state of the art by a statistically significant margin.

READ FULL TEXT
research
03/08/2021

Improving Global Adversarial Robustness Generalization With Adversarially Trained GAN

Convolutional neural networks (CNNs) have achieved beyond human-level ac...
research
09/15/2019

Wasserstein Diffusion Tikhonov Regularization

We propose regularization strategies for learning discriminative models ...
research
10/21/2017

ADA: A Game-Theoretic Perspective on Data Augmentation for Object Detection

The use of random perturbations of ground truth data, such as random tra...
research
08/18/2021

Semantic Perturbations with Normalizing Flows for Improved Generalization

Data augmentation is a widely adopted technique for avoiding overfitting...
research
02/28/2021

Adversarial Information Bottleneck

The information bottleneck (IB) principle has been adopted to explain de...
research
01/17/2021

Removing Undesirable Feature Contributions Using Out-of-Distribution Data

Several data augmentation methods deploy unlabeled-in-distribution (UID)...
research
06/19/2020

A general framework for defining and optimizing robustness

Robustness of neural networks has recently attracted a great amount of i...

Please sign up or login with your details

Forgot password? Click here to reset