Removing Undesirable Feature Contributions Using Out-of-Distribution Data

by   Saehyung Lee, et al.

Several data augmentation methods deploy unlabeled-in-distribution (UID) data to bridge the gap between the training and inference of neural networks. However, these methods have clear limitations in terms of availability of UID data and dependence of algorithms on pseudo-labels. Herein, we propose a data augmentation method to improve generalization in both adversarial and standard learning by using out-of-distribution (OOD) data that are devoid of the abovementioned issues. We show how to improve generalization theoretically using OOD data in each learning scenario and complement our theoretical analysis with experiments on CIFAR-10, CIFAR-100, and a subset of ImageNet. The results indicate that undesirable features are shared even among image data that seem to have little correlation from a human point of view. We also present the advantages of the proposed method through comparison with other data augmentation methods, which can be used in the absence of UID data. Furthermore, we demonstrate that the proposed method can further improve the existing state-of-the-art adversarial training.



There are no comments yet.


page 1

page 2

page 3

page 4


Adversarial Vertex Mixup: Toward Better Adversarially Robust Generalization

Adversarial examples cause neural networks to produce incorrect outputs ...

Dynamic Data Augmentation with Gating Networks

Data augmentation is a technique to improve the generalization ability o...

Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustness

Adversarial data augmentation has shown promise for training robust deep...

Stochastic Batch Augmentation with An Effective Distilled Dynamic Soft Label Regularizer

Data augmentation have been intensively used in training deep neural net...

Boosting Mapping Functionality of Neural Networks via Latent Feature Generation based on Reversible Learning

This paper addresses a boosting method for mapping functionality of neur...

Improving Non-native Word-level Pronunciation Scoring with Phone-level Mixup Data Augmentation and Multi-source Information

Deep learning-based pronunciation scoring models highly rely on the avai...

Transfer Incremental Learning using Data Augmentation

Deep learning-based methods have reached state of the art performances, ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.