Piecewise Linear Neural Network verification: A comparative study

11/01/2017
by   Rudy Bunel, et al.
0

The success of Deep Learning and its potential use in many important safety- critical applications has motivated research on formal verification of Neural Network (NN) models. Despite the reputation of learned NN models to behave as black boxes and the theoretical hardness of proving their properties, researchers have been successful in verifying some classes of models by exploiting their piecewise linear structure. Unfortunately, most of these approaches test their algorithms without comparison with other approaches. As a result, the pros and cons of the different algorithms are not well understood. Motivated by the need to accelerate progress in this very important area, we investigate the trade-offs of a number of different approaches based on Mixed Integer Programming, Satisfiability Modulo Theory, as well as a novel method based on the Branch-and-Bound framework. We also propose a new data set of benchmarks, in addition to a collection of pre- viously released testcases that can be used to compare existing methods. Our analysis not only allows a comparison to be made between different strategies, the comparison of results from different solvers also revealed implementation bugs in published methods. We expect that the availability of our benchmark and the analysis of the different approaches will allow researchers to develop and evaluate promising approaches for making progress on this important topic.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/14/2019

Branch and Bound for Piecewise Linear Neural Network Verification

The success of Deep Learning and its potential use in many safety-critic...
research
10/05/2019

Testing and verification of neural-network-based safety-critical control software: A systematic literature review

Context: Neural Network (NN) algorithms have been successfully adopted i...
research
12/13/2021

Geometric Path Enumeration for Equivalence Verification of Neural Networks

As neural networks (NNs) are increasingly introduced into safety-critica...
research
04/19/2019

Reward Potentials for Planning with Learned Neural Network Transition Models

Optimal planning with respect to learned neural network (NN) models in c...
research
07/21/2022

CheckINN: Wide Range Neural Network Verification in Imandra (Extended)

Neural networks are increasingly relied upon as components of complex sa...
research
01/14/2021

Scaling the Convex Barrier with Active Sets

Tight and efficient neural network bounding is of critical importance fo...

Please sign up or login with your details

Forgot password? Click here to reset