Adversarial Training and Provable Robustness: A Tale of Two Objectives

by   Jiameng Fan, et al.

We propose a principled framework that combines adversarial training and provable robustness verification for training certifiably robust neural networks. We formulate the training problem as a joint optimization problem with both empirical and provable robustness objectives and develop a novel gradient-descent technique that can eliminate bias in stochastic multi-gradients. We perform both theoretical analysis on the convergence of the proposed technique and experimental comparison with state-of-the-arts. Results on MNIST and CIFAR-10 show that our method can match or outperform prior approaches for provable l infinity robustness.



page 16


An SDE Framework for Adversarial Training, with Convergence and Robustness Analysis

Adversarial training has gained great popularity as one of the most effe...

Inductive Bias of Gradient Descent based Adversarial Training on Separable Data

Adversarial training is a principled approach for training robust neural...

Large-scale Stochastic Optimization of NDCG Surrogates for Deep Learning with Provable Convergence

NDCG, namely Normalized Discounted Cumulative Gain, is a widely used ran...

Provable Robustness of Adversarial Training for Learning Halfspaces with Noise

We analyze the properties of adversarial training for learning adversari...

Bayesian Inference with Certifiable Adversarial Robustness

We consider adversarial training of deep neural networks through the len...

A Multiclass Boosting Framework for Achieving Fast and Provable Adversarial Robustness

Alongside the well-publicized accomplishments of deep neural networks th...

Skew Orthogonal Convolutions

Training convolutional neural networks with a Lipschitz constraint under...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.