ZoPE: A Fast Optimizer for ReLU Networks with Low-Dimensional Inputs

by   Christopher A. Strong, et al.

Deep neural networks often lack the safety and robustness guarantees needed to be deployed in safety critical systems. Formal verification techniques can be used to prove input-output safety properties of networks, but when properties are difficult to specify, we rely on the solution to various optimization problems. In this work, we present an algorithm called ZoPE that solves optimization problems over the output of feedforward ReLU networks with low-dimensional inputs. The algorithm eagerly splits the input space, bounding the objective using zonotope propagation at each step, and improves computational efficiency compared to existing mixed integer programming approaches. We demonstrate how to formulate and solve three types of optimization problems: (i) minimization of any convex function over the output space, (ii) minimization of a convex function over the output of two networks in series with an adversarial perturbation in the layer between them, and (iii) maximization of the difference in output between two networks. Using ZoPE, we observe a 25× speedup on property 1 of the ACAS Xu neural network verification benchmark and an 85× speedup on a set of linear optimization problems. We demonstrate the versatility of the optimizer in analyzing networks by projecting onto the range of a generative adversarial network and visualizing the differences between a compressed and uncompressed network.



There are no comments yet.


page 8

page 9


Reachable Set Computation and Safety Verification for Neural Networks with ReLU Activations

Neural networks have been widely used to solve complex real-world proble...

ReLU Networks as Surrogate Models in Mixed-Integer Linear Programs

We consider the embedding of piecewise-linear deep neural networks (ReLU...

Effective Formal Verification of Neural Networks using the Geometry of Linear Regions

Neural Networks (NNs) have increasingly apparent safety implications com...

ReluDiff: Differential Verification of Deep Neural Networks

As deep neural networks are increasingly being deployed in practice, the...

Fast Falsification of Neural Networks using Property Directed Testing

Neural networks are now extensively used in perception, prediction and c...

Finding Input Characterizations for Output Properties in ReLU Neural Networks

Deep Neural Networks (DNNs) have emerged as a powerful mechanism and are...

Acceleration techniques for optimization over trained neural network ensembles

We study optimization problems where the objective function is modeled t...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.