Dissecting Neural ODEs

02/19/2020
by   Stefano Massaroli, et al.
14

Continuous deep learning architectures have recently re-emerged as variants of Neural Ordinary Differential Equations (Neural ODEs). The infinite-depth approach offered by these models theoretically bridges the gap between deep learning and dynamical systems; however, deciphering their inner working is still an open challenge and most of their applications are currently limited to the inclusion as generic black-box modules. In this work, we "open the box" and offer a system-theoretic perspective, including state augmentation strategies and robustness, with the aim of clarifying the influence of several design choices on the underlying dynamics. We also introduce novel architectures: among them, a Galerkin-inspired depth-varying parameter model and neural ODEs with data-controlled vector fields.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 2

page 8

page 18

page 19

09/20/2020

TorchDyn: A Neural Differential Equations Library

Continuous-depth learning has recently emerged as a novel perspective on...
02/22/2021

Neural Delay Differential Equations

Neural Ordinary Differential Equations (NODEs), a framework of continuou...
01/04/2022

Neural Piecewise-Constant Delay Differential Equations

Continuous-depth neural networks, such as the Neural Ordinary Differenti...
02/22/2021

Learning Contact Dynamics using Physically Structured Neural Networks

Learning physically structured representations of dynamical systems that...
09/11/2021

Structure-preserving Sparse Identification of Nonlinear Dynamics for Data-driven Modeling

Discovery of dynamical systems from data forms the foundation for data-d...
07/19/2020

Hypersolvers: Toward Fast Continuous-Depth Models

The infinite-depth paradigm pioneered by Neural ODEs has launched a rena...
06/12/2020

On Second Order Behaviour in Augmented Neural ODEs

Neural Ordinary Differential Equations (NODEs) are a new class of models...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.