Partial Differential Equations is All You Need for Generating Neural Architectures – A Theory for Physical Artificial Intelligence Systems

by   Ping Guo, et al.
Xi'an Jiaotong-Liverpool University
Beijing Normal University
Harbin Institute of Technology

In this work, we generalize the reaction-diffusion equation in statistical physics, Schrödinger equation in quantum mechanics, Helmholtz equation in paraxial optics into the neural partial differential equations (NPDE), which can be considered as the fundamental equations in the field of artificial intelligence research. We take finite difference method to discretize NPDE for finding numerical solution, and the basic building blocks of deep neural network architecture, including multi-layer perceptron, convolutional neural network and recurrent neural networks, are generated. The learning strategies, such as Adaptive moment estimation, L-BFGS, pseudoinverse learning algorithms and partial differential equation constrained optimization, are also presented. We believe it is of significance that presented clear physical image of interpretable deep neural networks, which makes it be possible for applying to analog computing device design, and pave the road to physical artificial intelligence.


page 2

page 3

page 4

page 5

page 6

page 7

page 13

page 14


Learning To Solve Differential Equations Across Initial Conditions

Recently, there has been a lot of interest in using neural networks for ...

Some observations on partial differential equations in Barron and multi-layer spaces

We use explicit representation formulas to show that solutions to certai...

Solving the Schrodinger equation with genetic algorithms: a practical approach

The Schrodinger equation is one of the most important equations in physi...

Dimensionally Consistent Preconditioning for Saddle-Point Problems

The preconditioned iterative solution of large-scale saddle-point system...

Connections between Numerical Algorithms for PDEs and Neural Networks

We investigate numerous structural connections between numerical algorit...

Deep Learning with Data Dependent Implicit Activation Function

Though deep neural networks (DNNs) achieve remarkable performances in ma...

Please sign up or login with your details

Forgot password? Click here to reset