PDE-constrained Models with Neural Network Terms: Optimization and Global Convergence

05/18/2021
by   Justin Sirignano, et al.
0

Recent research has used deep learning to develop partial differential equation (PDE) models in science and engineering. The functional form of the PDE is determined by a neural network, and the neural network parameters are calibrated to available data. Calibration of the embedded neural network can be performed by optimizing over the PDE. Motivated by these applications, we rigorously study the optimization of a class of linear elliptic PDEs with neural network terms. The neural network parameters in the PDE are optimized using gradient descent, where the gradient is evaluated using an adjoint PDE. As the number of parameters become large, the PDE and adjoint PDE converge to a non-local PDE system. Using this limit PDE system, we are able to prove convergence of the neural network-PDE to a global minimum during the optimization. The limit PDE system contains a non-local linear operator whose eigenvalues are positive but become arbitrarily small. The lack of a spectral gap for the eigenvalues poses the main challenge for the global convergence proof. Careful analysis of the spectral decomposition of the coupled PDE and adjoint PDE system is required. Finally, we use this adjoint method to train a neural network model for an application in fluid mechanics, in which the neural network functions as a closure model for the Reynolds-averaged Navier-Stokes (RANS) equations. The RANS neural network model is trained on several datasets for turbulent channel flow and is evaluated out-of-sample at different Reynolds numbers.

READ FULL TEXT
research
05/10/2023

Global Convergence of Deep Galerkin and PINNs Methods for Solving Partial Differential Equations

Numerically solving high-dimensional partial differential equations (PDE...
research
11/20/2019

DPM: A deep learning PDE augmentation method (with application to large-eddy simulation)

Machine learning for scientific applications faces the challenge of limi...
research
07/12/2022

MultiShape: A Spectral Element Method, with Applications to Dynamic Density Functional Theory and PDE-Constrained Optimization

A numerical framework is developed to solve various types of PDEs on com...
research
06/10/2022

How much can one learn from a single solution of a PDE?

Linear evolution PDE ∂_t u(x,t) = -ℒ u, where ℒ is a strongly elliptic o...
research
03/11/2021

Frame-independent vector-cloud neural network for nonlocal constitutive modelling on arbitrary grids

Constitutive models are widely used for modelling complex systems in sci...
research
05/28/2023

StEik: Stabilizing the Optimization of Neural Signed Distance Functions and Finer Shape Representation

We present new insights and a novel paradigm (StEik) for learning implic...
research
09/30/2022

Towards Multi-spatiotemporal-scale Generalized PDE Modeling

Partial differential equations (PDEs) are central to describing complex ...

Please sign up or login with your details

Forgot password? Click here to reset