Direct Domain Decomposition Method (D3M) for Finite Element Electromagnetic Computations

by   Javad Moshfegh, et al.

An exact arithmetic, memory efficient direct solution method for finite element method (FEM) computations is outlined. Unlike conventional black-box or low-rank direct solvers that are opaque to the underlying physical problem, the proposed method leverages physical insights at every stage of the development through a new symmetric domain decomposition method (DDM) with one set of Lagrange multipliers. Comparisons with state-of-the-art exact direct solvers on electrically large problems suggest up to 10 times less memory and better run-time complexity while maintaining the same accuracy.



There are no comments yet.


page 1

page 2


Direct Solution of FEM Models: Are Sparse Direct Solvers the Best Strategy?

A brief summary of direct solution approaches for finite element methods...

Parallel Direct Domain Decomposition Methods (D3M) for Finite Elements

A parallel direct solution approach based on domain decomposition method...

Efficient Direct Space-Time Finite Element Solvers for Parabolic Initial-Boundary Value Problems in Anisotropic Sobolev Spaces

We consider a space-time variational formulation of parabolic initial-bo...

The Statistical Finite Element Method

The finite element method (FEM) is one of the great triumphs of modern d...

Structural stochastic responses determination via a sample-based stochastic finite element method

This paper presents a new stochastic finite element method for computing...

Testing performance with and without Block Low Rank Compression in MUMPS and the new PaStiX 6.0 for JOREK nonlinear MHD simulations

The interface to the MUMPS solver was updated in the JOREK MHD code to s...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Modern computational electromagnetics FEM tools often rely on memory efficient iterative solvers such as Multigrid [References] and DDM [References] that may experience convergence difficulties near resonances or multi-scale problems, and lose efficiency at multiple excitation runs. Contrary, direct solvers such as MUMPS [References] or PARDISO are reliable but scale unfavorably and are hard to parallelize. Thus, recent trends in direct solvers [References] strive to reduce workload by leveraging low-rank approximations at the cost of accuracy and possibly reliability. Yet, these solvers are opaque to important underlying physics, leaving room for further improvements.

To achieve an efficient direct solver, one must start from scratch and attempt to leverage deep physical and numerical insights that may require re-formulating the BVP and FEM assembling, in addition to the symbolic and numeric factorization stages. This is critical not only to avoid internal resonances at all intermediate factorization separators, but also to produce numerically efficient matrix structures i.e. reduced size, block-wise sparse symmetric matrices.

This work achieves all these via a direct DDM (DM) framework. A set of auxiliary variables is used to cast a decomposed BVP that, after an initial reduction/elimination step, leads to an auxiliary blocked matrix that is suitable for factorization. To attain maximal performance, this matrix is factored with a special blocked LDL method with restricted Bunch-Kaufman pivoting [References].

The accuracy and performance of the proposed DM has been verified and tested in 3D scattering problems by perfect electric conductor (PEC) plates and dielectric spheres of progressively larger electrical sizes. The proposed DM solver requires less memory than MUMPS mainly due to the choice of structured separators and the absence of delayed pivots attributed to the interior resonance free formulation. An initial serial implementation of DM was up to two times slower than MUMPS for small problems but becomes competitive on problems larger than one million unknowns.

Ii Theory

Consider a computational domain decomposed into non-overlapping subdomain . For example, a decomposed problem with four domains is shown in Fig. 1. The decomposed BVP reads, :

where , and . is the restriction operator from domain to interface , and denotes the neighbor of domain .

Fig. 1: 2D decomposition problem with four domains.

After transforming to , casting the variational problem and expanding trial and testing function spaces gives


where is the diagonal blocked matrix of where is the FEM-ABC matrix for domain with loss or gain at the interfaces. is a blocked matrix of sparse matrices which maps the primal space to LM space. is the number of interfaces.

The reduced matrix is symmetric block-wise sparse but indefinite. Hence LDL factorization with symmetric partial pivoting a.k.a Bunch-Kaufmann LDL [References] can be used to save memory and CPU time. Since is a block-wise sparse matrix, we have modified the Bunch-Kaufman LDL factorization to its block restricted partial pivoted form. Each block in corresponds to a super-node of typical order . Therefore, DM consistently operates at the maximum performance region of Level 3 BLAS. The main steps of the proposed DM are:

  1. Generate dense domain matrices ,

  2. Assemble the block-wise sparse reduced matrix ,

  3. Reorder the clique graph of ,

  4. Symbolic factorize the reordered clique graph,

  5. Factorize with restricted B-K pivoting block LDL factorization (see algorithm 1),

  6. Forward/Backward substitute the reduced system for auxiliary unknowns,

  7. Recover primal unknowns.

The clique graph of blocked sparse matrix is reordered using METIS (same as MUMPS). Assuming that the clique graph has levels, the algorithm of block LDL factorization is given in Algorithm 1. A multifrontal version of block LDL can be used to further speed computations.

  for  do
     Dense LDL factorize
     for  do
        Dense Triangular solve for
        Compute work variable
        for  do
           Dense update
        end for
     end for
  end for
Algorithm 1 Block LDL factorization

Iii Numerical Results

First, the scattering of progressively larger PEC plates (from up to ) are considered. Computational complexity of factorization time and memory for these problems using the proposed DM and MUMPS are shown in Fig. 2. It is noted that a 3M unknown problem is solved with only 10 GB of RAM at full double precision accuracy.

Next, the scattering of progressively larger dielectric spheres is considered. Computational complexity of factorization time and memory for these problems using the proposed DM and MUMPS are shown in Fig. 3. Again, the proposed method uses more than 2.25 times less memory and surprisingly better time complexity than MUMPS. The relative residual error () of all runs using the proposed DM is around which was the same as MUMPS.

Fig. 2: Computational complexity for progressively larger PEC plates using the proposed DM and MUMPS: Factorization time (top); Factorization memory (bottom).
Fig. 3: Computational complexity for progressively larger dielectric spheres using the proposed DM and MUMPS: Factorization time (top); Factorization memory (bottom).


  • [1] J. Moshfegh, D. G. Makris, and M. N. Vouvakis,“Parallel Direct Domain Decomposition Methods (DM) for Finite Elements.” 2019 IEEE International Symposium on Antennas and Propagation and USNC-URSI Radio Science Meeting, pp. 777–778.
  • [2] Y. Zhu and A. C. Cangellaris, Multigrid finite element methods for electromagnetic field modeling. John Wiley & Sons, Vol. 28, 2006.
  • [3] A. Toselli and O. Widlund, Domain decomposition methods: algorithms and theory, Vol. 3. Berlin: Springer, 2005.
  • [4] P. R. Amestoy, I. S. Duff, J. Y. L’Excellent, and J. Koster, “A fully asynchronous multifrontal solver using distributed dynamic scheduling,” SIAM Journal on Matrix Analysis and Applications Vol. 23, No. 1, pp.15-41, 2001.
  • [5] P. G. Schmitz, and L. Ying, “A fast nested dissection solver for Cartesian 3D elliptic problems using hierarchical matrices,” Journal of Computational Physics Vol. 258, pp.227-245, 2014.
  • [6] J. R. Bunch, L. Kaufman, “Some stable methods for calculating inertia and solving symmetric linear systems,” Math. Comp., Vol. 31, No. 137, pp. 163–179, 1977.