Stability Analysis of Recurrent Neural Networks by IQC with Copositive Mutipliers

02/09/2022
by   Yoshio Ebihara, et al.
0

This paper is concerned with the stability analysis of the recurrent neural networks (RNNs) by means of the integral quadratic constraint (IQC) framework. The rectified linear unit (ReLU) is typically employed as the activation function of the RNN, and the ReLU has specific nonnegativity properties regarding its input and output signals. Therefore, it is effective if we can derive IQC-based stability conditions with multipliers taking care of such nonnegativity properties. However, such nonnegativity (linear) properties are hardly captured by the existing multipliers defined on the positive semidefinite cone. To get around this difficulty, we loosen the standard positive semidefinite cone to the copositive cone, and employ copositive multipliers to capture the nonnegativity properties. We show that, within the framework of the IQC, we can employ copositive multipliers (or their inner approximation) together with existing multipliers such as Zames-Falb multipliers and polytopic bounding multipliers, and this directly enables us to ensure that the introduction of the copositive multipliers leads to better (no more conservative) results. We finally illustrate the effectiveness of the IQC-based stability conditions with the copositive multipliers by numerical examples.

READ FULL TEXT
research
03/31/2021

Linear systems with neural network nonlinearities: Improved stability analysis via acausal Zames-Falb multipliers

In this paper, we analyze the stability of feedback interconnections of ...
research
04/10/2023

Approximation of Nonlinear Functionals Using Deep ReLU Networks

In recent years, functional neural networks have been proposed and studi...
research
01/30/2019

Generalized Tensor Models for Recurrent Neural Networks

Recurrent Neural Networks (RNNs) are very successful at solving challeng...
research
08/27/2019

Spectrum-based stability analysis and stabilization of a class of time-periodic time delay systems

We develop an eigenvalue-based approach for the stability assessment and...
research
11/20/2020

Low-Dimensional Manifolds Support Multiplexed Integrations in Recurrent Neural Networks

We study the learning dynamics and the representations emerging in Recur...
research
02/18/2021

Verifying Probabilistic Specifications with Functional Lagrangians

We propose a general framework for verifying input-output specifications...
research
07/17/2020

The energy technique for the six-step BDF method

In combination with the Grenander–Szegö theorem, we observe that a relax...

Please sign up or login with your details

Forgot password? Click here to reset