DeepAI
Log In Sign Up

Unsupervised Deep Learning Algorithm for PDE-based Forward and Inverse Problems

04/10/2019
by   Leah Bar, et al.
0

We propose a neural network-based algorithm for solving forward and inverse problems for partial differential equations in unsupervised fashion. The solution is approximated by a deep neural network which is the minimizer of a cost function, and satisfies the PDE, boundary conditions, and additional regularizations. The method is mesh free and can be easily applied to an arbitrary regular domain. We focus on 2D second order elliptical system with non-constant coefficients, with application to Electrical Impedance Tomography.

READ FULL TEXT VIEW PDF

page 6

page 8

page 9

page 10

page 11

page 12

04/28/2021

Consensus ADMM for Inverse Problems Governed by Multiple PDE Models

The Alternating Direction Method of Multipliers (ADMM) provides a natura...
02/26/2020

Numerical Solution of Inverse Problems by Weak Adversarial Networks

We consider a weak adversarial network approach to numerically solve a c...
02/22/2021

NTopo: Mesh-free Topology Optimization using Implicit Neural Representations

Recent advances in implicit neural representations show great promise wh...
06/12/2021

Graph-based Prior and Forward Models for Inverse Problems on Manifolds with Boundaries

This paper develops manifold learning techniques for the numerical solut...
04/20/2020

Deep Learning for One-dimensional Consolidation

Neural networks with physical governing equations as constraints have re...

1 Introduction

Inverse problems in partial differential equations are fundamental in science and mathematics with wide applications in medical imaging, signal processing, computer vision, remote sensing, electromagnetism and more. Classical methods such as finite differences, finite volume and finite elements are numerical discretization-based methods where the domain is divided into a uniform grid or polygon mesh. The differential equation is then reduced to a system of algebraic equations. These methods may have some limitations: the solution is numeric and may suffer from high condition number, highly dependent on the discretization and even the second derivative is sensitive to noise.

In the last few years, deep learning and neural network-based algorithms are extensively used in pattern recognition, image processing, computer vision and more. Recently, the deep learning approach had been adopted to the field of PDEs as well by converting the problem into a machine learning one. In

Supervised learning, the network maps an input to an output based on example input-output pairs. This strategy is used in inverse problems, where the input to the network is a set of observations/measurements (e.g. x-ray tomography, ultrasound) and the output is the set of parameters of interest (tissue density etc.) [4, 8, 9]. Unsupervised learning on the other hand is a self-learning mechanism where the natural structure presents within a set of data points is inferred.

Algorithms for forward and inverse problems in partial differential equations via unsupervised learning were recently introduced. The indirect approach utilizes a neural network as a component in the solution. Li et al. [7] for example, proposed the NETT (Network Tikhonov) approach to inverse problems. NETT considers regularized solutions having small value of a regularizer defined by a trained neural network. Khoo and Ying [6] introduced a novel neural network architecture, SwitchNet, for solving the wave equation based inverse scattering problems via providing maps between the scatterers and the scattered field. Han et al. [5] developed a deep learning-based approach that can handle general high-dimensional parabolic PDEs. To this end, the PDEs are reformulated using backward stochastic differential equations. The latter is solved by a temporal discretization and the gradient of the unknown solution at each time step is approximated by neural network.

Direct algorithms solve the forward problem PDEs by directly approximating the solution with a deep neural network. The network parameters are determined by the optimization of a cost function such that the optimal solution satisfies the PDE, boundary conditions and initial conditions. Chiaramonte and Kiener [2] addressed the forward problem by constructing a one layer network which satisfies the PDE within the domain. The boundary conditions were analytically integrated in the cost function. They demonstrated their algorithm on the Laplace and hyperbolic conservation law PDEs. Sirignano and Spiliopoulos [13] proposed a deep learning forward problem solver for high dimensional PDEs. Their algorithm was demonstrated on the American option free-boundary equation. Raissi et al. [11] focused on continuous time models and solved the Burgers and Shrödinger equations.

In this work we focus on the forward and inverse PDEs problems via a direct unsupervised method. Our key contributions are three fold: (1) in the forward part we extend the standard -based fidelity term in the cost function by adding -like norm. Moreover, (2) some regularization terms which impose a-priori knowledge on the solution can be easily incorporated. (3) An important feature of our construction is the ability to handle free-form domain in a mesh free manner. We demonstrate our algorithm by a second order elliptic equation, in particular the Electrical Impedance Tomography (EIT) application.

2 Mathematical Formulation

Let be a bounded open and connected subset of , and be any given symmetric positive definite matrix of functions for . Let be any given n-tuple of functions and let be any given function. A second order operator is said to be in divergence form, if acting on some has the form

(1)

where we use the Einstein summation convention. Consider the partial differential problem with Dirichlet boundary conditions

(2)

The forward problem solves given the coefficients while the inverse problem determines the coefficients set given .

The proposed algorithm approximates the solutions in both problems by neural networks such that the networks are parameterized by , and the input to the network is . Figure 1 depicts a network architecture of in . The network consists of few fully connected layers with tanh activation and linear sum in the last layer.

The network is trained to satisfy the PDE with the boundary conditions by minimizing a cost function. In the forward problem

(3)

and in the inverse problem

(4)

The first two terms enforce the solution to satisfy the equation. The first term minimizes the error in sense while the second term minimizes the maximal error. This term is important since the term only forces the equation up to a set of zero measure. The

term takes care of possible outliers. The third term imposes boundary conditions and the last term is a regularizer which can be tailored to the application. There are few advantages of this setting. First, the solutions are smooth analytic functions and are therefore

analytically differentiable. In addition, this framework enables setting of a prior knowledge on the solution by designing the regularizers and . Lastly, the training procedure is mesh free. In the sequel, we use random points in the domain and its boundary in the course of the optimization of (3) and  (4). This means that the solution does not depend upon a coordinate mesh and we can also define in principle an arbitrary regular domain .

Figure 1: Network architecture: the point serves as an input and as the output.

3 Application to Electrical Impedance Tomography

Let us address a special case of (1),

(5)

We assume that , which guarantees existence and uniqueness of a solution  [3].

The elliptical system (5) was addressed by Siltanen et al. [12] in the context of Electrical Impedance Tomography (EIT) which is a reconstruction method for the inverse conductivity problem. The function stands for the electrical conductivity density, and is the electrical potential. An electrical current

is applied on electrodes on the surface , where is the angle in polar coordinate system along the domain boundary and is the normal unit. The resulting voltage is measured through the electrodes. The conductivity is determined from the knowledge of the Dirichlet-to-Neumann map or voltage-to-current map

using the D-bar method [10].

We demonstrate our framework by solving the forward and inverse problem of (5) which is a first step towards a full tomography. Following Mueller and Siltanen [10], we simulate the voltage measurement by the Finite Element Method (FEM) given two variants of a conductivity phantom on the unit disc. We calculate the FEM solution with different triangle mesh densities such that finer meshes do not improve the numerical solution.

With our suggested method, the forward problem determines the electrical potential in the whole domain , while the inverse problem uses the approximated and calculates the conductivity given that . Throughout the paper we use three different electrical currents where , see Figure 2.

Figure 2: Electrical current for

4 Forward Problem

In the forward problem the conductivity and boundary conditions are given for random points set , with sets size of and respectively. A neural network having the architecture shown in Figure 1 approximates . Let

(6)

The cost function (3) is then rewritten as

(7)

The first term is the norm of the differential operator, the second term is a relaxed version of the infinity norm where we take the mean value of the top-K values of . The third term imposes the boundary conditions and the last term serves as a regularizer of the network parameters.

The network was trained with layers having , and neurons. The algorithm was implemented by TensorFlow [1] using the ADAM optimizer which is a variant of the SGD algorithm. We used batch size= and a decaying learning rate starting at corresponding to . The learning rate was factored by every epochs. The algorithm parameters were set to , , , , and .

The first phantom is shown in Figure 3. The background has conductivity and the circle has conductivity . The original piecewise constant function was slightly smoothed by a Gaussian kernel.

Figure 3: The conductivity of phantom

Figure 4 summarizes the forward problem results for currents and . The top row is the FEM solution which is referred to as ground truth. The middle row depicts the outcome of the trained network, and the bottom row shows the relative error ,

Mean square errors and PSNR are indicated in the figures’ caption.

(a) FEM,
(b) FEM,
(d) Proposed,
(e) Proposed,
(f) Proposed,
(g) Error,
(h) Error,
(i) Error,
(c) FEM,
(c) FEM,
Figure 4: Top: ground truth (FEM) of for given phantom . Middle: reconstruction by the proposed method. MSE = , PSNR=. Bottom: relative error

Figure 5 shows the derivative of with respect to . The top row is the finite difference approximation of the FEM result. The middle row is the analytical derivative of our result, and the bottom row shows the relative error.

(a) FEM,
(b) FEM,
(d) Proposed,
(e) Proposed,
(f) Proposed,
(g) Error,
(h) Error,
(i) Error,
(c) FEM,
(c) FEM,
Figure 5: Top: ground truth (FEM) of for given phantom . Middle: reconstruction by the proposed method. MSE = PSNR = . Bottom: relative error.

We further demonstrate the effect of the norm in the cost function. The left and right images of Figure 6 stand for the derivative of phantom without and with the term respectively. Clearly, this additional norm yields better reconstruction both visually (sharper circle edges) and quantitatively.

Figure 6: The derivative of phantom for . Left: without norm. MSE = 3.93e-8, PSNR = 32.61. Right: with norm. MSE = 2.84e-8, PSNR = 34.02

We repeated the experiment with an additional phantom, see Figure 7. The impedance values associated with the background, ellipses and circle were set so , and . In this case , learning rate= and all other parameters as before.

Figure 7: The conductivity of phantom

The results for both and are presented in figures 8 and 9 respectively.

(a) FEM,
(b) FEM,
(d) Proposed,
(e) Proposed,
(f) Proposed,
(g) Error,
(h) Error,
(i) Error,
(c) FEM,
(c) FEM,
Figure 8: Top: ground truth (FEM) of for given phantom . Middle: reconstruction by the proposed method. MSE = PSNR=. Bottom: relative error.
(a) FEM,
(b) FEM,
(d) Proposed,
(e) Proposed,
(f) Proposed,
(g) Error,
(h) Error,
(i) Error,
(c) FEM,
(c) FEM,
Figure 9: Top: ground truth (FEM) of for given phantom . Middle: reconstruction by the proposed method. MSE= PSNR=. Bottom: relative error.

5 Inverse Problem

In the inverse problem, the electrical potential is known while is unknown. Since we have a network which approximates , we can evaluate it at any point . The objective function (4) then takes the form

(8)

As in the forward problem, the first two terms enforce to satisfy the PDE, where is defined in (6). The third term imposes the boundary conditions, and the fourth regularizes the network parameters. The last term is the total variation regularization () which promotes the solution towards a piecewise constant solution. The network architecture and other parameters are as in the forward problem except for and . Conductivity reconstructions are shown in figures 10 and 11 with

(a)
(b)
(c)
Figure 10: Reconstructed of phantom . MSE = PSNR =
(a)
(b)
(c)
Figure 11: Reconstructed of phantom . MSE = PSNR =

6 Discussion

Deep networks by their nature use compositions of simple functions such as matrix multiplication and non-linear activations like sigmoid or tanh. This structure (i) enables the approximation of an arbitrary function and (ii) is inherently differentiable. The network architecture dictates the number of degrees of freedom which in turn enables the expressibility of complex functions. In this work we present a unified framework for the solution of forward and backward problems in partial differential equations. The algorithm relies on direct approximation of the unknown function by a neural network which yields an

analytical smooth solution in a predefined domain. The network is trained to satisfy the PDE and boundary conditions in an unsupervised fashion by the minimization of a cost function. The optimization procedure depends on random points set within the domain and its boundary. The problem is therefore mesh free with free-form domain. We introduce a cost function which is composed of both and fidelity terms and additional regularizers. The algorithm is demonstrated by an elliptic system in applied to Electrical Impedance Tomography for both forward and inverse problems. Promising results were achieved for complex and non monotonic functions. This framework is general and opens up a wide range of applications and extensions for further research.

References