Stiff Neural Ordinary Differential Equations

03/29/2021
by   Suyong Kim, et al.
0

Neural Ordinary Differential Equations (ODE) are a promising approach to learn dynamic models from time-series data in science and engineering applications. This work aims at learning Neural ODE for stiff systems, which are usually raised from chemical kinetic modeling in chemical and biological systems. We first show the challenges of learning neural ODE in the classical stiff ODE systems of Robertson's problem and propose techniques to mitigate the challenges associated with scale separations in stiff systems. We then present successful demonstrations in stiff systems of Robertson's problem and an air pollution problem. The demonstrations show that the usage of deep networks with rectified activations, proper scaling of the network outputs as well as loss functions, and stabilized gradient calculations are the key techniques enabling the learning of stiff neural ODE. The success of learning stiff neural ODE opens up possibilities of using neural ODEs in applications with widely varying time-scales, like chemical dynamics in energy conversion, environmental engineering, and the life sciences.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

11/09/2020

Stiff-PINN: Physics-Informed Neural Network for Stiff Chemical Kinetics

Recently developed physics-informed neural network (PINN) has achieved s...
10/28/2020

Parameterized Neural Ordinary Differential Equations: Applications to Computational Physics Problems

This work proposes an extension of neural ordinary differential equation...
12/03/2020

ChemNODE: A Neural Ordinary Differential Equations Approach for Chemical Kinetics Solvers

The main bottleneck when performing computational fluid dynamics (CFD) s...
10/13/2020

Modeling Atmospheric Data and Identifying Dynamics: Temporal Data-Driven Modeling of Air Pollutants

Atmospheric modelling has recently experienced a surge with the advent o...
05/24/2021

Autonomous Kinetic Modeling of Biomass Pyrolysis using Chemical Reaction Neural Networks

Modeling the burning processes of biomass such as wood, grass, and crops...
08/06/2019

Model inference for Ordinary Differential Equations by parametric polynomial kernel regression

Model inference for dynamical systems aims to estimate the future behavi...
07/18/2019

Can Machine Learning Identify Governing Laws For Dynamics in Complex Engineered Systems ? : A Study in Chemical Engineering

Machine learning recently has been used to identify the governing equati...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.