Implementation and (Inverse Modified) Error Analysis for implicitly-templated ODE-nets

by   Aiqing Zhu, et al.

We focus on learning unknown dynamics from data using ODE-nets templated on implicit numerical initial value problem solvers. First, we perform Inverse Modified error analysis of the ODE-nets using unrolled implicit schemes for ease of interpretation. It is shown that training an ODE-net using an unrolled implicit scheme returns a close approximation of an Inverse Modified Differential Equation (IMDE). In addition, we establish a theoretical basis for hyper-parameter selection when training such ODE-nets, whereas current strategies usually treat numerical integration of ODE-nets as a black box. We thus formulate an adaptive algorithm which monitors the level of error and adapts the number of (unrolled) implicit solution iterations during the training process, so that the error of the unrolled approximation is less than the current learning loss. This helps accelerate training, while maintaining accuracy. Several numerical experiments are performed to demonstrate the advantages of the proposed algorithm compared to nonadaptive unrollings, and validate the theoretical analysis. We also note that this approach naturally allows for incorporating partially known physical terms in the equations, giving rise to what is termed “gray box" identification.


page 1

page 2

page 3

page 4


Error analysis based on inverse modified differential equations for discovery of dynamics using linear multistep methods and deep learning

Along with the practical success of the discovery of dynamics using deep...

On Numerical Integration in Neural Ordinary Differential Equations

The combination of ordinary differential equations and neural networks, ...

Inverse modified differential equations for discovery of dynamics

We introduce inverse modified differential equations (IMDEs) to contribu...

An Error Analysis Framework for Neural Network Modeling of Dynamical Systems

We propose a theoretical framework for investigating a modeling error ca...

Sparse Identification of Truncation Errors

This work presents a data-driven approach to the identification of spati...

A New Implicit-Explicit Local Method to Capture Stiff Behavior with COVID-19 Outbreak Application

In this paper, a new implicit-explicit local method with an arbitrary or...

Please sign up or login with your details

Forgot password? Click here to reset