Characterizing possible failure modes in physics-informed neural networks

09/02/2021
by   Aditi S. Krishnapriyan, et al.
6

Recent work in scientific machine learning has developed so-called physics-informed neural network (PINN) models. The typical approach is to incorporate physical domain knowledge as soft constraints on an empirical loss function and use existing machine learning methodologies to train the model. We demonstrate that, while existing PINN methodologies can learn good models for relatively trivial problems, they can easily fail to learn relevant physical phenomena even for simple PDEs. In particular, we analyze several distinct situations of widespread physical interest, including learning differential equations with convection, reaction, and diffusion operators. We provide evidence that the soft regularization in PINNs, which involves differential operators, can introduce a number of subtle problems, including making the problem ill-conditioned. Importantly, we show that these possible failure modes are not due to the lack of expressivity in the NN architecture, but that the PINN's setup makes the loss landscape very hard to optimize. We then describe two promising solutions to address these failure modes. The first approach is to use curriculum regularization, where the PINN's loss term starts from a simple PDE regularization, and becomes progressively more complex as the NN gets trained. The second approach is to pose the problem as a sequence-to-sequence learning task, rather than learning to predict the entire space-time at once. Extensive testing shows that we can achieve up to 1-2 orders of magnitude lower error with these methods as compared to regular PINN training.

READ FULL TEXT

page 8

page 12

page 15

page 20

page 21

research
04/10/2023

iPINNs: Incremental learning for Physics-informed neural networks

Physics-informed neural networks (PINNs) have recently become a powerful...
research
05/05/2022

Lagrangian PINNs: A causality-conforming solution to failure modes of physics-informed neural networks

Physics-informed neural networks (PINNs) leverage neural-networks to fin...
research
01/13/2020

Understanding and mitigating gradient pathologies in physics-informed neural networks

The widespread use of neural networks across different scientific domain...
research
07/03/2023

Learning Generic Solutions for Multiphase Transport in Porous Media via the Flux Functions Operator

Traditional numerical schemes for simulating fluid flow and transport in...
research
07/08/2022

Adaptive Self-supervision Algorithms for Physics-informed Neural Networks

Physics-informed neural networks (PINNs) incorporate physical knowledge ...
research
10/23/2022

Less Emphasis on Difficult Layer Regions: Curriculum Learning for Singularly Perturbed Convection-Diffusion-Reaction Problems

Although Physics-Informed Neural Networks (PINNs) have been successfully...
research
12/10/2020

Comparison of Update and Genetic Training Algorithms in a Memristor Crossbar Perceptron

Memristor-based computer architectures are becoming more attractive as a...

Please sign up or login with your details

Forgot password? Click here to reset