HyperLoRA for PDEs

by   Ritam Majumdar, et al.

Physics-informed neural networks (PINNs) have been widely used to develop neural surrogates for solutions of Partial Differential Equations. A drawback of PINNs is that they have to be retrained with every change in initial-boundary conditions and PDE coefficients. The Hypernetwork, a model-based meta learning technique, takes in a parameterized task embedding as input and predicts the weights of PINN as output. Predicting weights of a neural network however, is a high-dimensional regression problem, and hypernetworks perform sub-optimally while predicting parameters for large base networks. To circumvent this issue, we use a low ranked adaptation (LoRA) formulation to decompose every layer of the base network into low-ranked tensors and use hypernetworks to predict the low-ranked tensors. Despite the reduced dimensionality of the resulting weight-regression problem, LoRA-based Hypernetworks violate the underlying physics of the given task. We demonstrate that the generalization capabilities of LoRA-based hypernetworks drastically improve when trained with an additional physics-informed loss component (HyperPINN) to satisfy the governing differential equations. We observe that LoRA-based HyperPINN training allows us to learn fast solutions for parameterized PDEs like Burger's equation and Navier Stokes: Kovasznay flow, while having an 8x reduction in prediction parameters on average without compromising on accuracy when compared to all other baselines.


A certified wavelet-based physics-informed neural network for the solution of parameterized partial differential equations

Physics Informed Neural Networks (PINNs) have frequently been used for t...

Symbolic Regression for PDEs using Pruned Differentiable Programs

Physics-informed Neural Networks (PINNs) have been widely used to obtain...

Laplace HypoPINN: Physics-Informed Neural Network for hypocenter localization and its predictive uncertainty

Several techniques have been proposed over the years for automatic hypoc...

Learning-based solutions to nonlinear hyperbolic PDEs: Empirical insights on generalization errors

We study learning weak solutions to nonlinear hyperbolic partial differe...

Meta Learning of Interface Conditions for Multi-Domain Physics-Informed Neural Networks

Physics-informed neural networks (PINNs) are emerging as popular mesh-fr...

When Do Extended Physics-Informed Neural Networks (XPINNs) Improve Generalization?

Physics-informed neural networks (PINNs) have become a popular choice fo...

Please sign up or login with your details

Forgot password? Click here to reset