Deep Neural Networks motivated by Partial Differential Equations

04/12/2018
by   Lars Ruthotto, et al.
0

Partial differential equations (PDEs) are indispensable for modeling many physical phenomena and also commonly used for solving image processing tasks. In the latter area, PDE-based approaches interpret image data as discretizations of multivariate functions and the output of image processing algorithms as solutions to certain PDEs. Posing image processing problems in the infinite dimensional setting provides powerful tools for their analysis and solution. Over the last three decades, the reinterpretation of classical image processing tasks through the PDE lens has been creating multiple celebrated approaches that benefit a vast area of tasks including image segmentation, denoising, registration, and reconstruction. In this paper, we establish a new PDE-interpretation of deep convolution neural networks (CNN) that are commonly used for learning tasks involving speech, image, and video data. Our interpretation includes convolution residual neural networks (ResNet), which are among the most promising approaches for tasks such as image classification having improved the state-of-the-art performance in prestigious benchmark challenges. Despite their recent successes, deep ResNets still face some critical challenges associated with their design, immense computational costs and memory requirements, and lack of understanding of their reasoning. Guided by well-established PDE theory, we derive three new ResNet architectures that fall two new classes: parabolic and hyperbolic CNNs. We demonstrate how PDE theory can provide new insights and algorithms for deep learning and demonstrate the competitiveness of three new CNN architectures using numerical experiments.

READ FULL TEXT

page 2

page 6

research
06/28/2020

Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theory

Deep learning has significantly revolutionized the design of numerical a...
research
12/16/2019

VarNet: Variational Neural Networks for the Solution of Partial Differential Equations

In this paper we propose a new model-based unsupervised learning method,...
research
08/04/2021

PDE-GCN: Novel Architectures for Graph Neural Networks Motivated by Partial Differential Equations

Graph neural networks are increasingly becoming the go-to approach in va...
research
05/18/2020

PDE constraints on smooth hierarchical functions computed by neural networks

Neural networks are versatile tools for computation, having the ability ...
research
02/16/2021

Shape-Tailored Deep Neural Networks

We present Shape-Tailored Deep Neural Networks (ST-DNN). ST-DNN extend c...
research
05/20/2018

Low-Cost Parameterizations of Deep Convolution Neural Networks

The main computational cost in the training of and prediction with Convo...
research
09/08/2022

Clifford Neural Layers for PDE Modeling

Partial differential equations (PDEs) see widespread use in sciences and...

Please sign up or login with your details

Forgot password? Click here to reset