Does Data Augmentation Lead to Positive Margin?

05/08/2019
by   Shashank Rajput, et al.
0

Data augmentation (DA) is commonly used during model training, as it significantly improves test error and model robustness. DA artificially expands the training set by applying random noise, rotations, crops, or even adversarial perturbations to the input data. Although DA is widely used, its capacity to provably improve robustness is not fully understood. In this work, we analyze the robustness that DA begets by quantifying the margin that DA enforces on empirical risk minimizers. We first focus on linear separators, and then a class of nonlinear models whose labeling is constant within small convex hulls of data points. We present lower bounds on the number of augmented data points required for non-zero margin, and show that commonly used DA techniques may only introduce significant margin after adding exponentially many points to the data set.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/16/2022

A Data-Augmentation Is Worth A Thousand Samples: Exact Quantification From Analytical Augmented Sample Moments

Data-Augmentation (DA) is known to improve performance across tasks and ...
research
04/12/2023

Towards Understanding How Data Augmentation Works with Imbalanced Data

Data augmentation forms the cornerstone of many modern machine learning ...
research
05/25/2023

Visualizing data augmentation in deep speaker recognition

Visualization is of great value in understanding the internal mechanisms...
research
11/07/2019

Data transforming augmentation for heteroscedastic models

Data augmentation (DA) turns seemingly intractable computational problem...
research
03/08/2021

Consistency Regularization for Adversarial Robustness

Adversarial training (AT) is currently one of the most successful method...
research
09/18/2021

Asynchronous and Distributed Data Augmentation for Massive Data Settings

Data augmentation (DA) algorithms are widely used for Bayesian inference...

Please sign up or login with your details

Forgot password? Click here to reset