Transformer for Partial Differential Equations' Operator Learning

05/26/2022
by   Zijie Li, et al.
50

Data-driven learning of partial differential equations' solution operators has recently emerged as a promising paradigm for approximating the underlying solutions. The solution operators are usually parameterized by deep learning models that are built upon problem-specific inductive biases. An example is a convolutional or a graph neural network that exploits the local grid structure where functions' values are sampled. The attention mechanism, on the other hand, provides a flexible way to implicitly exploit the patterns within inputs, and furthermore, relationship between arbitrary query locations and inputs. In this work, we present an attention-based framework for data-driven operator learning, which we term Operator Transformer (OFormer). Our framework is built upon self-attention, cross-attention, and a set of point-wise multilayer perceptrons (MLPs), and thus it makes few assumptions on the sampling pattern of the input function or query locations. We show that the proposed framework is competitive on standard benchmark problems and can flexibly be adapted to randomly sampled input.

READ FULL TEXT

page 16

page 18

page 24

page 26

page 27

research
02/19/2022

Function-valued RKHS-based Operator Learning for Differential Equations

Recently, a steam of works seek for solving a family of partial differen...
research
02/28/2023

GNOT: A General Neural Operator Transformer for Operator Learning

Learning partial differential equations' (PDEs) solution operators is an...
research
05/31/2021

Choose a Transformer: Fourier or Galerkin

In this paper, we apply the self-attention from the state-of-the-art Tra...
research
01/04/2022

Learning Operators with Coupled Attention

Supervised operator learning is an emerging machine learning paradigm wi...
research
05/27/2023

Scalable Transformer for PDE Surrogate Modeling

Transformer has shown state-of-the-art performance on various applicatio...
research
05/24/2023

Generative diffusion learning for parametric partial differential equations

We develop a class of data-driven generative models that approximate the...
research
09/07/2019

GMLS-Nets: A framework for learning from unstructured data

Data fields sampled on irregularly spaced points arise in many applicati...

Please sign up or login with your details

Forgot password? Click here to reset