Path classification by stochastic linear recurrent neural networks

08/06/2021
by   Wiebke Bartolomaeus, et al.
0

We investigate the functioning of a classifying biological neural network from the perspective of statistical learning theory, modelled, in a simplified setting, as a continuous-time stochastic recurrent neural network (RNN) with identity activation function. In the purely stochastic (robust) regime, we give a generalisation error bound that holds with high probability, thus showing that the empirical risk minimiser is the best-in-class hypothesis. We show that RNNs retain a partial signature of the paths they are fed as the unique information exploited for training and classification tasks. We argue that these RNNs are easy to train and robust and back these observations with numerical experiments on both synthetic and real data. We also exhibit a trade-off phenomenon between accuracy and robustness.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/06/2018

Sliced Recurrent Neural Networks

Recurrent neural networks have achieved great success in many NLP tasks....
research
05/23/2021

Spectral Pruning for Recurrent Neural Networks

Pruning techniques for neural networks with a recurrent architecture, su...
research
06/22/2020

Understanding Recurrent Neural Networks Using Nonequilibrium Response Theory

Recurrent neural networks (RNNs) are brain-inspired models widely used i...
research
08/24/2016

Recurrent Neural Networks With Limited Numerical Precision

Recurrent Neural Networks (RNNs) produce state-of-art performance on man...
research
11/25/2022

Minimal Width for Universal Property of Deep RNN

A recurrent neural network (RNN) is a widely used deep-learning network ...
research
05/19/2017

Online Signature Verification using Recurrent Neural Network and Length-normalized Path Signature

Inspired by the great success of recurrent neural networks (RNNs) in seq...
research
04/07/2023

Theoretical Conditions and Empirical Failure of Bracket Counting on Long Sequences with Linear Recurrent Networks

Previous work has established that RNNs with an unbounded activation fun...

Please sign up or login with your details

Forgot password? Click here to reset