ZoPE: A Fast Optimizer for ReLU Networks with Low-Dimensional Inputs

06/09/2021
by   Christopher A. Strong, et al.
0

Deep neural networks often lack the safety and robustness guarantees needed to be deployed in safety critical systems. Formal verification techniques can be used to prove input-output safety properties of networks, but when properties are difficult to specify, we rely on the solution to various optimization problems. In this work, we present an algorithm called ZoPE that solves optimization problems over the output of feedforward ReLU networks with low-dimensional inputs. The algorithm eagerly splits the input space, bounding the objective using zonotope propagation at each step, and improves computational efficiency compared to existing mixed integer programming approaches. We demonstrate how to formulate and solve three types of optimization problems: (i) minimization of any convex function over the output space, (ii) minimization of a convex function over the output of two networks in series with an adversarial perturbation in the layer between them, and (iii) maximization of the difference in output between two networks. Using ZoPE, we observe a 25× speedup on property 1 of the ACAS Xu neural network verification benchmark and an 85× speedup on a set of linear optimization problems. We demonstrate the versatility of the optimizer in analyzing networks by projecting onto the range of a generative adversarial network and visualizing the differences between a compressed and uncompressed network.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 8

page 9

12/21/2017

Reachable Set Computation and Safety Verification for Neural Networks with ReLU Activations

Neural networks have been widely used to solve complex real-world proble...
07/06/2019

ReLU Networks as Surrogate Models in Mixed-Integer Linear Programs

We consider the embedding of piecewise-linear deep neural networks (ReLU...
06/18/2020

Effective Formal Verification of Neural Networks using the Geometry of Linear Regions

Neural Networks (NNs) have increasingly apparent safety implications com...
01/10/2020

ReluDiff: Differential Verification of Deep Neural Networks

As deep neural networks are increasingly being deployed in practice, the...
04/26/2021

Fast Falsification of Neural Networks using Property Directed Testing

Neural networks are now extensively used in perception, prediction and c...
03/09/2020

Finding Input Characterizations for Output Properties in ReLU Neural Networks

Deep Neural Networks (DNNs) have emerged as a powerful mechanism and are...
12/13/2021

Acceleration techniques for optimization over trained neural network ensembles

We study optimization problems where the objective function is modeled t...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.