Convex Sets of Robust Recurrent Neural Networks

04/11/2020
by   Max Revay, et al.
0

Recurrent neural networks (RNNs) are a class of nonlinear dynamical systems often used to model sequence-to-sequence maps. RNNs have been shown to have excellent expressive power but lack stability or robustness guarantees that would be necessary for safety-critical applications. In this paper we formulate convex sets of RNNs with guaranteed stability and robustness properties. The guarantees are derived using differential IQC methods and can ensure contraction (global exponential stability of all solutions) and bounds on incremental l2 gain (the Lipschitz constant of the learnt sequence-to-sequence mapping). An implicit model structure is employed to construct a jointly-convex representation of an RNN and its certificate of stability or robustness. We prove that the proposed model structure includes all previously-proposed convex sets of contracting RNNs as special cases, and also includes all stable linear dynamical systems. We demonstrate the utility of the proposed model class in the context of nonlinear system identification.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/22/2019

Contracting Implicit Recurrent Neural Networks: Stable Models with Improved Trainability

Stability of recurrent models is closely linked with trainability, gener...
research
04/13/2021

Recurrent Equilibrium Networks: Unconstrained Learning of Stable and Robust Dynamical Models

This paper introduces recurrent equilibrium networks (RENs), a new class...
research
06/25/2020

On Lyapunov Exponents for RNNs: Understanding Information Propagation Using Dynamical Systems Tools

Recurrent neural networks (RNNs) have been successfully applied to a var...
research
03/02/2021

On the Memory Mechanism of Tensor-Power Recurrent Models

Tensor-power (TP) recurrent model is a family of non-linear dynamical sy...
research
12/20/2021

Adversarially Robust Stability Certificates can be Sample-Efficient

Motivated by bridging the simulation to reality gap in the context of sa...
research
11/08/2021

On the Stochastic Stability of Deep Markov Models

Deep Markov models (DMM) are generative models that are scalable and exp...
research
05/10/2023

Frequency-Supported Neural Networks for Nonlinear Dynamical System Identification

Neural networks are a very general type of model capable of learning var...

Please sign up or login with your details

Forgot password? Click here to reset