Negative Data Augmentation

by   Abhishek Sinha, et al.

Data augmentation is often used to enlarge datasets with synthetic samples generated in accordance with the underlying data distribution. To enable a wider range of augmentations, we explore negative data augmentation strategies (NDA)that intentionally create out-of-distribution samples. We show that such negative out-of-distribution samples provide information on the support of the data distribution, and can be leveraged for generative modeling and representation learning. We introduce a new GAN training objective where we use NDA as an additional source of synthetic data for the discriminator. We prove that under suitable conditions, optimizing the resulting objective still recovers the true data distribution but can directly bias the generator towards avoiding samples that lack the desired structure. Empirically, models trained with our method achieve improved conditional/unconditional image generation along with improved anomaly detection capabilities. Further, we incorporate the same negative data augmentation strategy in a contrastive learning framework for self-supervised representation learning on images and videos, achieving improved performance on downstream image classification, object detection, and action recognition tasks. These results suggest that prior knowledge on what does not constitute valid data is an effective form of weak supervision across a range of unsupervised learning tasks.


page 2

page 15

page 16


OMASGAN: Out-of-Distribution Minimum Anomaly Score GAN for Sample Generation on the Boundary

Generative models trained in an unsupervised manner may set high likelih...

Learning Representational Invariances for Data-Efficient Action Recognition

Data augmentation is a ubiquitous technique for improving image classifi...

Instance-Conditioned GAN Data Augmentation for Representation Learning

Data augmentation has become a crucial component to train state-of-the-a...

Data Augmentation for Seizure Prediction with Generative Diffusion Model

Objective: Seizure prediction is of great importance to improve the life...

A Guide for Practical Use of ADMG Causal Data Augmentation

Data augmentation is essential when applying Machine Learning in small-d...

Chosen methods of improving small object recognition with weak recognizable features

Many object detection models struggle with several problematic aspects o...

Please sign up or login with your details

Forgot password? Click here to reset