Fast and Complete: Enabling Complete Neural Network Verification with Rapid and Massively Parallel Incomplete Verifiers

11/27/2020
by   Kaidi Xu, et al.
0

Formal verification of neural networks (NNs) is a challenging and important problem. Existing efficient complete solvers typically require the branch-and-bound (BaB) process, which splits the problem domain into sub-domains and solves each sub-domain using faster but weaker incomplete verifiers, such as Linear Programming (LP) on linearly relaxed sub-domains. In this paper, we propose to use the backward mode linear relaxation based perturbation analysis (LiRPA) to replace LP during the BaB process, which can be efficiently implemented on the typical machine learning accelerators such as GPUs and TPUs. However, unlike LP, LiRPA when applied naively can produce much weaker bounds and even cannot check certain conflicts of sub-domains during splitting, making the entire procedure incomplete after BaB. To address these challenges, we apply a fast gradient based bound tightening procedure combined with batch splits and the design of minimal usage of LP bound procedure, enabling us to effectively use LiRPA on the accelerator hardware for the challenging complete NN verification problem and significantly outperform LP-based approaches. On a single GPU, we demonstrate an order of magnitude speedup compared to existing LP-based approaches.

READ FULL TEXT

page 1

page 2

page 3

page 4

03/11/2021

Beta-CROWN: Efficient Bound Propagation with Per-neuron Split Constraints for Complete and Incomplete Neural Network Verification

Recent works in neural network verification show that cheap incomplete v...
06/18/2012

LPQP for MAP: Putting LP Solvers to Better Use

MAP inference for general energy functions remains a challenging problem...
10/22/2020

Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming

Convex relaxations have emerged as a promising approach for verifying de...
01/11/2021

New commodity representations for multicommodity network flow problems: An application to the fixed-charge network design problem

When solving hard multicommodity network flow problems using an LP-based...
11/04/2015

Train and Test Tightness of LP Relaxations in Structured Prediction

Structured prediction is used in areas such as computer vision and natur...
06/15/2021

Improving Search by Utilizing State Information in OPTIC Planners Compilation to LP

Automated planners are computer tools that allow autonomous agents to ma...
04/30/2022

Complete Verification via Multi-Neuron Relaxation Guided Branch-and-Bound

State-of-the-art neural network verifiers are fundamentally based on one...