Robustness Certificates for Implicit Neural Networks: A Mixed Monotone Contractive Approach

12/10/2021
by   Saber Jafarpour, et al.
0

Implicit neural networks are a general class of learning models that replace the layers in traditional feedforward models with implicit algebraic equations. Compared to traditional learning models, implicit networks offer competitive performance and reduced memory consumption. However, they can remain brittle with respect to input adversarial perturbations. This paper proposes a theoretical and computational framework for robustness verification of implicit neural networks; our framework blends together mixed monotone systems theory and contraction theory. First, given an implicit neural network, we introduce a related embedded network and show that, given an ℓ_∞-norm box constraint on the input, the embedded network provides an ℓ_∞-norm box overapproximation for the output of the given network. Second, using ℓ_∞-matrix measures, we propose sufficient conditions for well-posedness of both the original and embedded system and design an iterative algorithm to compute the ℓ_∞-norm box robustness margins for reachability and classification problems. Third, of independent value, we propose a novel relative classifier variable that leads to tighter bounds on the certified adversarial robustness in classification problems. Finally, we perform numerical simulations on a Non-Euclidean Monotone Operator Network (NEMON) trained on the MNIST dataset. In these simulations, we compare the accuracy and run time of our mixed monotone contractive approach with the existing robustness verification approaches in the literature for estimating the certified adversarial robustness.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/08/2022

Robust Training and Verification of Implicit Neural Networks: A Non-Euclidean Contractive Approach

This paper proposes a theoretical and computational framework for traini...
research
04/01/2022

Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networks

We use interval reachability analysis to obtain robustness guarantees fo...
research
06/06/2021

Robust Implicit Networks via Non-Euclidean Contractions

Implicit neural networks, a.k.a., deep equilibrium networks, are a class...
research
04/07/2023

Contraction-Guided Adaptive Partitioning for Reachability Analysis of Neural Network Controlled Systems

In this paper, we present a contraction-guided adaptive partitioning alg...
research
06/15/2020

Monotone operator equilibrium networks

Implicit-depth models such as Deep Equilibrium Networks have recently be...
research
09/19/2022

State-driven Implicit Modeling for Sparsity and Robustness in Neural Networks

Implicit models are a general class of learning models that forgo the hi...
research
10/30/2022

FI-ODE: Certified and Robust Forward Invariance in Neural ODEs

We study how to certifiably enforce forward invariance properties in neu...

Please sign up or login with your details

Forgot password? Click here to reset