AutoDO: Robust AutoAugment for Biased Data with Label Noise via Scalable Probabilistic Implicit Differentiation

03/10/2021
by   Denis Gudovskiy, et al.
0

AutoAugment has sparked an interest in automated augmentation methods for deep learning models. These methods estimate image transformation policies for train data that improve generalization to test data. While recent papers evolved in the direction of decreasing policy search complexity, we show that those methods are not robust when applied to biased and noisy data. To overcome these limitations, we reformulate AutoAugment as a generalized automated dataset optimization (AutoDO) task that minimizes the distribution shift between test data and distorted train dataset. In our AutoDO model, we explicitly estimate a set of per-point hyperparameters to flexibly change distribution of train data. In particular, we include hyperparameters for augmentation, loss weights, and soft-labels that are jointly estimated using implicit differentiation. We develop a theoretical probabilistic interpretation of this framework using Fisher information and show that its complexity scales linearly with the dataset size. Our experiments on SVHN, CIFAR-10/100, and ImageNet classification show up to 9.3 label noise compared to prior methods and, importantly, up to 36.6 underrepresented SVHN classes.

READ FULL TEXT
research
11/01/2017

Data, Depth, and Design: Learning Reliable Models for Melanoma Screening

State of the art on melanoma screening evolved rapidly in the last two y...
research
10/31/2019

Confident Learning: Estimating Uncertainty in Dataset Labels

Learning exists in the context of data, yet notions of confidence typica...
research
07/12/2021

Fine-Grained AutoAugmentation for Multi-Label Classification

Data augmentation is a commonly used approach to improving the generaliz...
research
03/08/2021

Size-Invariant Graph Representations for Graph Classification Extrapolations

In general, graph representation learning methods assume that the test a...
research
11/06/2019

Optimizing Millions of Hyperparameters by Implicit Differentiation

We propose an algorithm for inexpensive gradient-based hyperparameter op...
research
12/17/2020

DecAug: Out-of-Distribution Generalization via Decomposed Feature Representation and Semantic Augmentation

While deep learning demonstrates its strong ability to handle independen...
research
09/28/2022

Label Distribution Learning via Implicit Distribution Representation

In contrast to multi-label learning, label distribution learning charact...

Please sign up or login with your details

Forgot password? Click here to reset