HiDeNN-PGD: reduced-order hierarchical deep learning neural networks

by   Lei Zhang, et al.

This paper presents a proper generalized decomposition (PGD) based reduced-order model of hierarchical deep-learning neural networks (HiDeNN). The proposed HiDeNN-PGD method keeps both advantages of HiDeNN and PGD methods. The automatic mesh adaptivity makes the HiDeNN-PGD more accurate than the finite element method (FEM) and conventional PGD, using a fraction of the FEM degrees of freedom. The accuracy and convergence of the method have been studied theoretically and numerically, with a comparison to different methods, including FEM, PGD, HiDeNN and Deep Neural Networks. In addition, we theoretically showed that the PGD converges to FEM at increasing modes, and the PGD error is a direct sum of the FEM error and the mode reduction error. The proposed HiDeNN-PGD performs high accuracy with orders of magnitude fewer degrees of freedom, which shows a high potential to achieve fast computations with a high level of accuracy for large-size engineering problems.


page 4

page 10

page 18

page 19

page 32


Degrees of Freedom in Deep Neural Networks

In this paper, we explore degrees of freedom in deep sigmoidal neural ne...

Computing leaky modes of optical fibers using a FEAST algorithm for polynomial eigenproblems

An efficient technique to solve polynomial eigenproblems is shown to res...

Balancing truncation and round-off errors in practical FEM: one-dimensional analysis

In finite element methods (FEMs), the accuracy of the solution cannot in...

Separation of scales and a thermodynamic description of feature learning in some CNNs

Deep neural networks (DNNs) are powerful tools for compressing and disti...

Decomposition of admissible functions in weighted coupled cell networks

This work makes explicit the degrees of freedom involved in modeling the...

How many degrees of freedom do we need to train deep networks: a loss landscape perspective

A variety of recent works, spanning pruning, lottery tickets, and traini...

Recovering missing CFD data for high-order discretizations using deep neural networks and dynamics learning

Data I/O poses a significant bottleneck in large-scale CFD simulations; ...