Neural Network Approximations of PDEs Beyond Linearity: Representational Perspective

10/21/2022
by   Tanya Marwah, et al.
0

A burgeoning line of research has developed deep neural networks capable of approximating the solutions to high dimensional PDEs, opening related lines of theoretical inquiry focused on explaining how it is that these models appear to evade the curse of dimensionality. However, most theoretical analyses thus far have been limited to linear PDEs. In this work, we take a step towards studying the representational power of neural networks for approximating solutions to nonlinear PDEs. We focus on a class of PDEs known as nonlinear elliptic variational PDEs, whose solutions minimize an Euler-Lagrange energy functional ℰ(u) = ∫_Ω L(∇ u) dx. We show that if composing a function with Barron norm b with L produces a function of Barron norm at most B_L b^p, the solution to the PDE can be ϵ-approximated in the L^2 sense by a function with Barron norm O((dB_L)^p^log(1/ϵ)). By a classical result due to Barron [1993], this correspondingly bounds the size of a 2-layer neural network needed to approximate the solution. Treating p, ϵ, B_L as constants, this quantity is polynomial in dimension, thus showing neural networks can evade the curse of dimensionality. Our proof technique involves neurally simulating (preconditioned) gradient in an appropriate Hilbert space, which converges exponentially fast to the solution of the PDE, and such that we can bound the increase of the Barron norm at each iterate. Our results subsume and substantially generalize analogous prior results for linear elliptic PDEs.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/03/2021

Parametric Complexity Bounds for Approximating PDEs with Neural Networks

Recent empirical results show that deep networks can approximate solutio...
research
06/14/2021

On the Representation of Solutions to Elliptic PDEs in Barron Spaces

Numerical solutions to high-dimensional partial differential equations (...
research
08/21/2023

Deep Learning of Delay-Compensated Backstepping for Reaction-Diffusion PDEs

Deep neural networks that approximate nonlinear function-to-function map...
research
10/30/2021

Approximation properties of Residual Neural Networks for Kolmogorov PDEs

In recent years residual neural networks (ResNets) as introduced by [He,...
research
11/10/2021

Deep ReLU neural network approximation of parametric and stochastic elliptic PDEs with lognormal inputs

We investigate non-adaptive methods of deep ReLU neural network approxim...
research
01/28/2023

Deep Operator Learning Lessens the Curse of Dimensionality for PDEs

Deep neural networks (DNNs) have seen tremendous success in many fields ...

Please sign up or login with your details

Forgot password? Click here to reset