mixup: Beyond Empirical Risk Minimization

10/25/2017
by   Hongyi Zhang, et al.
0

Large deep neural networks are powerful, but exhibit undesirable behaviors such as memorization and sensitivity to adversarial examples. In this work, we propose mixup, a simple learning principle to alleviate these issues. In essence, mixup trains a neural network on convex combinations of pairs of examples and their labels. By doing so, mixup regularizes the neural network to favor simple linear behavior in-between training examples. Our experiments on the ImageNet-2012, CIFAR-10, CIFAR-100, Google commands and UCI datasets show that mixup improves the generalization of state-of-the-art neural network architectures. We also find that mixup reduces the memorization of corrupt labels, increases the robustness to adversarial examples, and stabilizes the training of generative adversarial networks.

READ FULL TEXT

page 1

page 2

page 3

page 4

02/24/2021

Multiplicative Reweighting for Robust Neural Network Optimization

Deep neural networks are widespread due to their powerful performance. Y...
12/11/2014

Towards Deep Neural Network Architectures Robust to Adversarial Examples

Recent work has shown deep neural networks (DNNs) to be highly susceptib...
01/30/2019

HyperGAN: A Generative Model for Diverse, Performant Neural Networks

We introduce HyperGAN, a generative network that learns to generate all ...
09/15/2021

Adversarial Mixing Policy for Relaxing Locally Linear Constraints in Mixup

Mixup is a recent regularizer for current deep classification networks. ...
11/01/2021

Investigating the locality of neural network training dynamics

A fundamental quest in the theory of deep-learning is to understand the ...
02/17/2021

Improving Hierarchical Adversarial Robustness of Deep Neural Networks

Do all adversarial examples have the same consequences? An autonomous dr...
09/19/2019

Training Robust Deep Neural Networks via Adversarial Noise Propagation

Deep neural networks have been found vulnerable to noises like adversari...

Code Repositories

mixup

Implementation of the mixup training method


view repo

Please sign up or login with your details

Forgot password? Click here to reset