Structure Probing Neural Network Deflation

by   Yiqi Gu, et al.

Deep learning is a powerful tool for solving nonlinear differential equations, but usually, only the solution corresponding to the flattest local minimizer can be found due to the implicit regularization of stochastic gradient descent. This paper proposes Structure Probing Neural Network Deflation (SP-NND) to make deep learning capable of identifying multiple solutions that are ubiquitous and important in nonlinear physical models. First, we introduce deflation operators built with known solutions to make known solutions no longer local minimizers of the optimization energy landscape. Second, to facilitate the convergence to the desired local minimizer, a structure probing technique is proposed to obtain an initial guess close to the desired local minimizer. Together with neural network structures carefully designed in this paper, the new regularized optimization can converge to new solutions efficiently. Due to the mesh-free nature of deep learning, SP-NND is capable of solving high-dimensional problems on complicated domains with multiple solutions, while existing methods focus on merely one or two-dimensional regular domains and are more expensive than SP-NND in operation counts. Numerical experiments also demonstrate that SP-NND could find more solutions than exiting methods.


page 18

page 21

page 22


Deep Learning algorithms for solving high dimensional nonlinear Backward Stochastic Differential Equations

We study deep learning-based schemes for solving high dimensional nonlin...

JDNN: Jacobi Deep Neural Network for Solving Telegraph Equation

In this article, a new deep learning architecture, named JDNN, has been ...

A randomized Newton's method for solving differential equations based on the neural network discretization

We develop a randomized Newton's method for solving differential equatio...

Gauss Newton method for solving variational problems of PDEs with neural network discretizaitons

The numerical solution of differential equations using machine learning-...

The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems

We propose a deep learning based method, the Deep Ritz Method, for numer...

A SOM-based Gradient-Free Deep Learning Method with Convergence Analysis

As gradient descent method in deep learning causes a series of questions...

On some nonlocal models with radially symmetric interaction domains and the effect of domain truncation

Many nonlocal models have adopted a finite and radially symmetric nonloc...

Please sign up or login with your details

Forgot password? Click here to reset