Traversing the Local Polytopes of ReLU Neural Networks: A Unified Approach for Network Verification

11/17/2021
by   Shaojie Xu, et al.
9

Although neural networks (NNs) with ReLU activation functions have found success in a wide range of applications, their adoption in risk-sensitive settings has been limited by the concerns on robustness and interpretability. Previous works to examine robustness and to improve interpretability partially exploited the piecewise linear function form of ReLU NNs. In this paper, we explore the unique topological structure that ReLU NNs create in the input space, identifying the adjacency among the partitioned local polytopes and developing a traversing algorithm based on this adjacency. Our polytope traversing algorithm can be adapted to verify a wide range of network properties related to robustness and interpretability, providing an unified approach to examine the network behavior. As the traversing algorithm explicitly visits all local polytopes, it returns a clear and full picture of the network behavior within the traversed region. The time and space complexity of the traversing algorithm is determined by the number of a ReLU NN's partitioning hyperplanes passing through the traversing region.

READ FULL TEXT

page 3

page 7

research
05/09/2023

SkelEx and BoundEx: Natural Visualization of ReLU Neural Networks

Despite their limited interpretability, weights and biases are still the...
research
05/15/2023

ReLU soothes the NTK condition number and accelerates optimization for wide neural networks

Rectified linear unit (ReLU), as a non-linear activation function, is we...
research
05/20/2020

ReLU Code Space: A Basis for Rating Network Quality Besides Accuracy

We propose a new metric space of ReLU activation codes equipped with a t...
research
10/14/2021

Sound and Complete Neural Network Repair with Minimality and Locality Guarantees

We present a novel methodology for repairing neural networks that use Re...
research
06/01/2023

Learning Prescriptive ReLU Networks

We study the problem of learning optimal policy from a set of discrete t...
research
12/22/2020

Bounding the Complexity of Formally Verifying Neural Networks: A Geometric Approach

In this paper, we consider the computational complexity of formally veri...
research
06/30/2023

ReLU Neural Networks, Polyhedral Decompositions, and Persistent Homolog

A ReLU neural network leads to a finite polyhedral decomposition of inpu...

Please sign up or login with your details

Forgot password? Click here to reset