A Convex Relaxation Barrier to Tight Robustness Verification of Neural Networks

by   Hadi Salman, et al.

Verification of neural networks enables us to gauge their robustness against adversarial attacks. Verification algorithms fall into two categories: exact verifiers that run in exponential time and relaxed verifiers that are efficient but incomplete. In this paper, we unify all existing LP-relaxed verifiers, to the best of our knowledge, under a general convex relaxation framework. This framework works for neural networks with diverse architectures and nonlinearities and covers both primal and dual views of robustness verification. We further prove strong duality between the primal and dual problems under very mild conditions. Next, we perform large-scale experiments, amounting to more than 22 CPU-years, to obtain exact solution to the convex-relaxed problem that is optimal within our framework for ReLU networks. We find the exact solution does not significantly improve upon the gap between PGD and existing relaxed verifiers for various networks trained normally or robustly on MNIST and CIFAR datasets. Our results suggest there is an inherent barrier to tight verification for the large class of methods captured by our framework. We discuss possible causes of this barrier and potential future directions for bypassing it.


page 2

page 11

page 29


A Convex Relaxation Barrier to Tight Robust Verification of Neural Networks

Verification of neural networks enables us to gauge their robustness aga...

Improving the Tightness of Convex Relaxation Bounds for Training Certifiably Robust Classifiers

Convex relaxations are effective for training and certifying neural netw...

The Convex Relaxation Barrier, Revisited: Tightened Single-Neuron Relaxations for Neural Network Verification

We improve the effectiveness of propagation- and linear-optimization-bas...

Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming

Convex relaxations have emerged as a promising approach for verifying de...

Universal Approximation with Certified Networks

Training neural networks to be certifiably robust is a powerful defense ...

Efficient Exact Verification of Binarized Neural Networks

Concerned with the reliability of neural networks, researchers have deve...

Scaling the Convex Barrier with Active Sets

Tight and efficient neural network bounding is of critical importance fo...