Certifiable Distributional Robustness with Principled Adversarial Training

10/29/2017
by   Aman Sinha, et al.
0

Neural networks are vulnerable to adversarial examples and researchers have proposed many heuristic attack and defense mechanisms. We take the principled view of distributionally robust optimization, which guarantees performance under adversarial input perturbations. By considering a Lagrangian penalty formulation of perturbation of the underlying data distribution in a Wasserstein ball, we provide a training procedure that augments model parameter updates with worst-case perturbations of training data. For smooth losses, our procedure provably achieves moderate levels of robustness with little computational or statistical cost relative to empirical risk minimization. Furthermore, our statistical guarantees allow us to efficiently certify robustness for the population loss. We match or outperform heuristic approaches on supervised and reinforcement learning tasks.

READ FULL TEXT

page 10

page 12

page 17

page 18

research
06/09/2019

Beyond Adversarial Training: Min-Max Optimization in Adversarial Attack and Defense

The worst-case training principle that minimizes the maximal adversarial...
research
07/22/2020

Robust Machine Learning via Privacy/Rate-Distortion Theory

Robust machine learning formulations have emerged to address the prevale...
research
06/07/2023

Optimal Transport Model Distributional Robustness

Distributional robustness is a promising framework for training deep lea...
research
06/05/2020

Principled Learning Method for Wasserstein Distributionally Robust Optimization with Local Perturbations

Wasserstein distributionally robust optimization (WDRO) attempts to lear...
research
02/09/2021

Provable Defense Against Delusive Poisoning

Delusive poisoning is a special kind of attack to obstruct learning, whe...
research
02/10/2021

Bayesian Inference with Certifiable Adversarial Robustness

We consider adversarial training of deep neural networks through the len...
research
10/20/2021

Distributionally Robust Semi-Supervised Learning Over Graphs

Semi-supervised learning (SSL) over graph-structured data emerges in man...

Please sign up or login with your details

Forgot password? Click here to reset