Second-Order Provable Defenses against Adversarial Attacks

06/01/2020
by   Sahil Singla, et al.
7

A robustness certificate is the minimum distance of a given input to the decision boundary of the classifier (or its lower bound). For any input perturbations with a magnitude smaller than the certificate value, the classification output will provably remain unchanged. Exactly computing the robustness certificates for neural networks is difficult since it requires solving a non-convex optimization. In this paper, we provide computationally-efficient robustness certificates for neural networks with differentiable activation functions in two steps. First, we show that if the eigenvalues of the Hessian of the network are bounded, we can compute a robustness certificate in the l_2 norm efficiently using convex optimization. Second, we derive a computationally-efficient differentiable upper bound on the curvature of a deep network. We also use the curvature bound as a regularization term during the training of the network to boost its certified robustness. Putting these results together leads to our proposed Curvature-based Robustness Certificate (CRC) and Curvature-based Robust Training (CRT). Our numerical results show that CRT leads to significantly higher certified robust accuracy compared to interval-bound propagation (IBP) based training. We achieve certified robust accuracy 69.79%, 57.78% and 53.19% while IBP-based methods achieve 44.96%, 44.74% and 44.66% on 2,3 and 4 layer networks respectively on the MNIST-dataset.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/01/2019

Robustness Certificates Against Adversarial Examples for ReLU Networks

While neural networks have achieved high performance in different learni...
research
02/03/2023

Asymmetric Certified Robustness via Feature-Convex Neural Networks

Recent works have introduced input-convex neural networks (ICNNs) as lea...
research
06/14/2022

Flatten the Curve: Efficiently Training Low-Curvature Neural Networks

The highly non-linear nature of deep neural networks causes them to be s...
research
10/19/2018

On Extensions of CLEVER: A Neural Network Robustness Evaluation Algorithm

CLEVER (Cross-Lipschitz Extreme Value for nEtwork Robustness) is an Extr...
research
04/21/2020

Probabilistic Safety for Bayesian Neural Networks

We study probabilistic safety for Bayesian Neural Networks (BNNs) under ...
research
06/03/2019

Fast and Stable Interval Bounds Propagation for Training Verifiably Robust Models

We present an efficient technique, which allows to train classification ...
research
12/11/2020

DSRNA: Differentiable Search of Robust Neural Architectures

In deep learning applications, the architectures of deep neural networks...

Please sign up or login with your details

Forgot password? Click here to reset