DeepAI AI Chat
Log In Sign Up

Convergence of Adversarial Training in Overparametrized Networks

06/19/2019
by   Ruiqi Gao, et al.
0

Neural networks are vulnerable to adversarial examples, i.e. inputs that are imperceptibly perturbed from natural data and yet incorrectly classified by the network. Adversarial training, a heuristic form of robust optimization that alternates between minimization and maximization steps, has proven to be among the most successful methods to train networks that are robust against a pre-defined family of perturbations. This paper provides a partial answer to the success of adversarial training. When the inner maximization problem can be solved to optimality, we prove that adversarial training finds a network of small robust train loss. When the maximization problem is solved by a heuristic algorithm, we prove that adversarial training finds a network of small robust surrogate train loss. The analysis technique leverages recent work on the analysis of neural networks via Neural Tangent Kernel (NTK), combined with online-learning when the maximization is solved by a heuristic, and the expressiveness of the NTK kernel in the ℓ_∞-norm.

READ FULL TEXT

page 1

page 2

page 3

page 4

12/15/2021

On the Convergence and Robustness of Adversarial Training

Improving the robustness of deep neural networks (DNNs) to adversarial e...
12/01/2021

ℓ_∞-Robustness and Beyond: Unleashing Efficient Adversarial Training

Neural networks are vulnerable to adversarial attacks: adding well-craft...
11/17/2015

Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization

We propose a general framework for increasing local stability of Artific...
11/03/2018

Learning to Defense by Learning to Attack

Adversarial training provides a principled approach for training robust ...
07/08/2020

Fast Training of Deep Neural Networks Robust to Adversarial Perturbations

Deep neural networks are capable of training fast and generalizing well ...
10/16/2020

Learning Robust Algorithms for Online Allocation Problems Using Adversarial Training

We address the challenge of finding algorithms for online allocation (i....
06/17/2019

MixUp as Directional Adversarial Training

In this work, we explain the working mechanism of MixUp in terms of adve...