Beta-CROWN: Efficient Bound Propagation with Per-neuron Split Constraints for Complete and Incomplete Neural Network Verification

03/11/2021
by   Shiqi Wang, et al.
0

Recent works in neural network verification show that cheap incomplete verifiers such as CROWN, based upon bound propagations, can effectively be used in Branch-and-Bound (BaB) methods to accelerate complete verification, achieving significant speedups compared to expensive linear programming (LP) based techniques. However, they cannot fully handle the per-neuron split constraints introduced by BaB like LP verifiers do, leading to looser bounds and hurting their verification efficiency. In this work, we develop β-CROWN, a new bound propagation based method that can fully encode per-neuron splits via optimizable parameters β. When the optimizable parameters are jointly optimized in intermediate layers, β-CROWN has the potential of producing better bounds than typical LP verifiers with neuron split constraints, while being efficiently parallelizable on GPUs. Applied to the complete verification setting, β-CROWN is close to three orders of magnitude faster than LP-based BaB methods for robustness verification, and also over twice faster than state-of-the-art GPU-based complete verifiers with similar timeout rates. By terminating BaB early, our method can also be used for incomplete verification. Compared to the state-of-the-art semidefinite-programming (SDP) based verifier, we show a substantial leap forward by greatly reducing the gap between verified accuracy and empirical adversarial attack accuracy, from 35 MNIST network (ϵ=0.3), while being 47 times faster. Our code is available at https://github.com/KaidiXu/Beta-CROWN

READ FULL TEXT

page 1

page 2

page 3

page 4

11/27/2020

Fast and Complete: Enabling Complete Neural Network Verification with Rapid and Massively Parallel Incomplete Verifiers

Formal verification of neural networks (NNs) is a challenging and import...
04/30/2022

Complete Verification via Multi-Neuron Relaxation Guided Branch-and-Bound

State-of-the-art neural network verifiers are fundamentally based on one...
10/22/2020

Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming

Convex relaxations have emerged as a promising approach for verifying de...
06/15/2022

Can pruning improve certified robustness of neural networks?

With the rapid development of deep learning, the sizes of neural network...
07/15/2022

3DVerifier: Efficient Robustness Verification for 3D Point Cloud Models

3D point cloud models are widely applied in safety-critical scenes, whic...
04/01/2021

Towards Evaluating and Training Verifiably Robust Neural Networks

Recent works have shown that interval bound propagation (IBP) can be use...
06/29/2022

IBP Regularization for Verified Adversarial Robustness via Branch-and-Bound

Recent works have tried to increase the verifiability of adversarially t...