DeepAI AI Chat
Log In Sign Up

Robust Training and Verification of Implicit Neural Networks: A Non-Euclidean Contractive Approach

08/08/2022
by   Saber Jafarpour, et al.
Georgia Institute of Technology
The Regents of the University of California
0

This paper proposes a theoretical and computational framework for training and robustness verification of implicit neural networks based upon non-Euclidean contraction theory. The basic idea is to cast the robustness analysis of a neural network as a reachability problem and use (i) the ℓ_∞-norm input-output Lipschitz constant and (ii) the tight inclusion function of the network to over-approximate its reachable sets. First, for a given implicit neural network, we use ℓ_∞-matrix measures to propose sufficient conditions for its well-posedness, design an iterative algorithm to compute its fixed points, and provide upper bounds for its ℓ_∞-norm input-output Lipschitz constant. Second, we introduce a related embedded network and show that the embedded network can be used to provide an ℓ_∞-norm box over-approximation of the reachable sets of the original network. Moreover, we use the embedded network to design an iterative algorithm for computing the upper bounds of the original system's tight inclusion function. Third, we use the upper bounds of the Lipschitz constants and the upper bounds of the tight inclusion functions to design two algorithms for the training and robustness verification of implicit neural networks. Finally, we apply our algorithms to train implicit neural networks on the MNIST dataset and compare the robustness of our models with the models trained via existing approaches in the literature.

READ FULL TEXT

page 1

page 2

page 3

page 4

12/10/2021

Robustness Certificates for Implicit Neural Networks: A Mixed Monotone Contractive Approach

Implicit neural networks are a general class of learning models that rep...
04/01/2022

Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networks

We use interval reachability analysis to obtain robustness guarantees fo...
06/06/2021

Robust Implicit Networks via Non-Euclidean Contractions

Implicit neural networks, a.k.a., deep equilibrium networks, are a class...
03/22/2021

Performance Bounds for Neural Network Estimators: Applications in Fault Detection

We exploit recent results in quantifying the robustness of neural networ...
10/28/2018

RecurJac: An Efficient Recursive Algorithm for Bounding Jacobian Matrix of Neural Networks and Its Applications

The Jacobian matrix (or the gradient for single-output networks) is dire...
10/13/2022

Efficiently Computing Local Lipschitz Constants of Neural Networks via Bound Propagation

Lipschitz constants are connected to many properties of neural networks,...
03/03/2023

Convex Bounds on the Softmax Function with Applications to Robustness Verification

The softmax function is a ubiquitous component at the output of neural n...