Abstract Neural Networks

09/11/2020
by   Matthew Sotoudeh, et al.
0

Deep Neural Networks (DNNs) are rapidly being applied to safety-critical domains such as drone and airplane control, motivating techniques for verifying the safety of their behavior. Unfortunately, DNN verification is NP-hard, with current algorithms slowing exponentially with the number of nodes in the DNN. This paper introduces the notion of Abstract Neural Networks (ANNs), which can be used to soundly overapproximate DNNs while using fewer nodes. An ANN is like a DNN except weight matrices are replaced by values in a given abstract domain. We present a framework parameterized by the abstract domain and activation functions used in the DNN that can be used to construct a corresponding ANN. We present necessary and sufficient conditions on the DNN activation functions for the constructed ANN to soundly over-approximate the given DNN. Prior work on DNN abstraction was restricted to the interval domain and ReLU activation function. Our framework can be instantiated with other abstract domains such as octagons and polyhedra, as well as other activation functions such as Leaky ReLU, Sigmoid, and Hyperbolic Tangent.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/10/2023

DNN Verification, Reachability, and the Exponential Function Problem

Deep neural networks (DNNs) are increasingly being deployed to perform s...
research
10/19/2022

A new activation for neural networks and its approximation

Deep learning with deep neural networks (DNNs) has attracted tremendous ...
research
10/13/2019

Large Deviation Analysis of Function Sensitivity in Random Deep Neural Networks

Mean field theory has been successfully used to analyze deep neural netw...
research
02/13/2015

Abstract Learning via Demodulation in a Deep Neural Network

Inspired by the brain, deep neural networks (DNN) are thought to learn a...
research
12/27/2021

FitAct: Error Resilient Deep Neural Networks via Fine-Grained Post-Trainable Activation Functions

Deep neural networks (DNNs) are increasingly being deployed in safety-cr...
research
10/19/2015

Qualitative Projection Using Deep Neural Networks

Deep neural networks (DNN) abstract by demodulating the output of linear...
research
05/05/2023

Repairing Deep Neural Networks Based on Behavior Imitation

The increasing use of deep neural networks (DNNs) in safety-critical sys...

Please sign up or login with your details

Forgot password? Click here to reset