A neuron-wise subspace correction method for the finite neuron method

11/22/2022
by   Jongho Park, et al.
0

In this paper, we propose a novel algorithm called Neuron-wise Parallel Subspace Correction Method (NPSC) for training ReLU neural networks for numerical solution of partial differential equations (PDEs). Despite of extremely extensive research activities in applying neural networks for numerical PDEs, there is still a serious lack of training algorithms that can be used to obtain approximation with adequate accuracy. Based on recent results on the spectral properties of linear layers and landscape analysis for single neuron problems, we develop a special type of subspace correction method that deals with the linear layer and each neuron in the nonlinear layer separately. An optimal preconditioner that resolves the ill-conditioning of the linear layer is presented, so that the linear layer is trained in a uniform number of iterations with respect to the number of neurons. In each single neuron problem, a good local minimum is found by a superlinearly convergent algorithm, avoiding regions where the loss function is flat. Performance of the proposed method is demonstrated through numerical experiments for function approximation problems and PDEs.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/14/2021

Adaptive Two-Layer ReLU Neural Network: II. Ritz Approximation to Elliptic PDEs

In this paper, we study adaptive neuron enhancement (ANE) method for sol...
research
09/12/2022

Numerical approximation based on deep convolutional neural network for high-dimensional fully nonlinear merged PDEs and 2BSDEs

In this paper, we propose two efficient approximation methods to solve h...
research
07/31/2019

Neural networks-based backward scheme for fully nonlinear PDEs

We propose a numerical method for solving high dimensional fully nonline...
research
07/19/2021

Adaptive Two-Layer ReLU Neural Network: I. Best Least-squares Approximation

In this paper, we introduce adaptive neuron enhancement (ANE) method for...
research
06/01/2022

The effect of time discretization on the solution of parabolic PDEs with ANNs

We investigate the resolution of parabolic PDEs via Extreme Learning Mac...
research
12/09/2019

Deep Ritz revisited

Recently, progress has been made in the application of neural networks t...
research
03/27/2023

GPT-PINN: Generative Pre-Trained Physics-Informed Neural Networks toward non-intrusive Meta-learning of parametric PDEs

Physics-Informed Neural Network (PINN) has proven itself a powerful tool...

Please sign up or login with your details

Forgot password? Click here to reset