Adversarial Auto-Augment with Label Preservation: A Representation Learning Principle Guided Approach

11/02/2022
by   Kaiwen Yang, et al.
33

Data augmentation is a critical contributing factor to the success of deep learning but heavily relies on prior domain knowledge which is not always available. Recent works on automatic data augmentation learn a policy to form a sequence of augmentation operations, which are still pre-defined and restricted to limited options. In this paper, we show that a prior-free autonomous data augmentation's objective can be derived from a representation learning principle that aims to preserve the minimum sufficient information of the labels. Given an example, the objective aims at creating a distant "hard positive example" as the augmentation, while still preserving the original label. We then propose a practical surrogate to the objective that can be optimized efficiently and integrated seamlessly into existing methods for a broad class of machine learning tasks, e.g., supervised, semi-supervised, and noisy-label learning. Unlike previous works, our method does not require training an extra generative model but instead leverages the intermediate layer representations of the end-task model for generating data augmentations. In experiments, we show that our method consistently brings non-trivial improvements to the three aforementioned learning tasks from both efficiency and final performance, either or not combined with strong pre-defined augmentations, e.g., on medical images when domain knowledge is unavailable and the existing augmentation techniques perform poorly. Code is available at: https://github.com/kai-wen-yang/LPA3https://github.com/kai-wen-yang/LPA3.

READ FULL TEXT
research
05/22/2023

Tied-Augment: Controlling Representation Similarity Improves Data Augmentation

Data augmentation methods have played an important role in the recent ad...
research
10/20/2021

Improving Model Generalization by Agreement of Learned Representations from Data Augmentation

Data augmentation reduces the generalization error by forcing a model to...
research
02/17/2022

Graph Data Augmentation for Graph Machine Learning: A Survey

Data augmentation has recently seen increased interest in graph machine ...
research
08/23/2021

Jointly Learnable Data Augmentations for Self-Supervised GNNs

Self-supervised Learning (SSL) aims at learning representations of objec...
research
10/21/2022

Exploring Representation-Level Augmentation for Code Search

Code search, which aims at retrieving the most relevant code fragment fo...
research
05/27/2023

Toward Understanding Generative Data Augmentation

Generative data augmentation, which scales datasets by obtaining fake la...
research
08/28/2017

DeepPrior++: Improving Fast and Accurate 3D Hand Pose Estimation

DeepPrior is a simple approach based on Deep Learning that predicts the ...

Please sign up or login with your details

Forgot password? Click here to reset