DeepAI AI Chat
Log In Sign Up

Taylor-Lagrange Neural Ordinary Differential Equations: Toward Fast Training and Evaluation of Neural ODEs

01/14/2022
by   Franck Djeumou, et al.
0

Neural ordinary differential equations (NODEs) – parametrizations of differential equations using neural networks – have shown tremendous promise in learning models of unknown continuous-time dynamical systems from data. However, every forward evaluation of a NODE requires numerical integration of the neural network used to capture the system dynamics, making their training prohibitively expensive. Existing works rely on off-the-shelf adaptive step-size numerical integration schemes, which often require an excessive number of evaluations of the underlying dynamics network to obtain sufficient accuracy for training. By contrast, we accelerate the evaluation and the training of NODEs by proposing a data-driven approach to their numerical integration. The proposed Taylor-Lagrange NODEs (TL-NODEs) use a fixed-order Taylor expansion for numerical integration, while also learning to estimate the expansion's approximation error. As a result, the proposed approach achieves the same accuracy as adaptive step-size schemes while employing only low-order Taylor expansions, thus greatly reducing the computational cost necessary to integrate the NODE. A suite of numerical experiments, including modeling dynamical systems, image classification, and density estimation, demonstrate that TL-NODEs can be trained more than an order of magnitude faster than state-of-the-art approaches, without any loss in performance.

READ FULL TEXT

page 1

page 2

page 3

page 4

12/19/2019

Polynomial Neural Networks and Taylor maps for Dynamical Systems Simulation and Learning

The connection of Taylor maps and polynomial neural networks (PNN) to so...
02/28/2022

Neural Ordinary Differential Equations for Nonlinear System Identification

Neural ordinary differential equations (NODE) have been recently propose...
12/28/2021

An Error Analysis Framework for Neural Network Modeling of Dynamical Systems

We propose a theoretical framework for investigating a modeling error ca...
05/19/2017

Scalable Variational Inference for Dynamical Systems

Gradient matching is a promising tool for learning parameters and state ...
06/16/2020

Neural Optimal Control for Representation Learning

The intriguing connections recently established between neural networks ...
07/04/2021

Learning ODEs via Diffeomorphisms for Fast and Robust Integration

Advances in differentiable numerical integrators have enabled the use of...
06/08/2021

Incorporating NODE with Pre-trained Neural Differential Operator for Learning Dynamics

Learning dynamics governed by differential equations is crucial for pred...