DeepAI AI Chat
Log In Sign Up

Tractable Dendritic RNNs for Reconstructing Nonlinear Dynamical Systems

by   Manuel Brenner, et al.

In many scientific disciplines, we are interested in inferring the nonlinear dynamical system underlying a set of observed time series, a challenging task in the face of chaotic behavior and noise. Previous deep learning approaches toward this goal often suffered from a lack of interpretability and tractability. In particular, the high-dimensional latent spaces often required for a faithful embedding, even when the underlying dynamics lives on a lower-dimensional manifold, can hamper theoretical analysis. Motivated by the emerging principles of dendritic computation, we augment a dynamically interpretable and mathematically tractable piecewise-linear (PL) recurrent neural network (RNN) by a linear spline basis expansion. We show that this approach retains all the theoretically appealing properties of the simple PLRNN, yet boosts its capacity for approximating arbitrary nonlinear dynamical systems in comparatively low dimensions. We employ two frameworks for training the system, one combining back-propagation-through-time (BPTT) with teacher forcing, and another based on fast and scalable variational inference. We show that the dendritically expanded PLRNN achieves better reconstructions with fewer parameters and dimensions on various dynamical systems benchmarks and compares favorably to other methods, while retaining a tractable and interpretable structure.


Identifying nonlinear dynamical systems via generative recurrent neural networks with applications to fMRI

A major tenet in theoretical neuroscience is that cognitive and behavior...

Multimodal Teacher Forcing for Reconstructing Nonlinear Dynamical Systems

Many, if not most, systems of interest in science are naturally describe...

Identifying nonlinear dynamical systems from multi-modal time series data

Empirically observed time series in physics, biology, or medicine, are c...

Inferring Global Dynamics Using a Learning Machine

Given a segment of time series of a system at a particular set of parame...

Tensorized Transformer for Dynamical Systems Modeling

The identification of nonlinear dynamics from observations is essential ...

The geometry of integration in text classification RNNs

Despite the widespread application of recurrent neural networks (RNNs) a...

Learning High Dimensional Demonstrations Using Laplacian Eigenmaps

This article proposes a novel methodology to learn a stable robot contro...