A Lagrangian Dual-based Theory-guided Deep Neural Network

08/24/2020
by   Miao Rong, et al.
7

The theory-guided neural network (TgNN) is a kind of method which improves the effectiveness and efficiency of neural network architectures by incorporating scientific knowledge or physical information. Despite its great success, the theory-guided (deep) neural network possesses certain limits when maintaining a tradeoff between training data and domain knowledge during the training process. In this paper, the Lagrangian dual-based TgNN (TgNN-LD) is proposed to improve the effectiveness of TgNN. We convert the original loss function into a constrained form with fewer items, in which partial differential equations (PDEs), engineering controls (ECs), and expert knowledge (EK) are regarded as constraints, with one Lagrangian variable per constraint. These Lagrangian variables are incorporated to achieve an equitable tradeoff between observation data and corresponding constraints, in order to improve prediction accuracy, and conserve time and computational resources adjusted by an ad-hoc procedure. To investigate the performance of the proposed method, the original TgNN model with a set of optimized weight values adjusted by ad-hoc procedures is compared on a subsurface flow problem, with their L2 error, R square (R2), and computational time being analyzed. Experimental results demonstrate the superiority of the Lagrangian dual-based TgNN.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

page 5

page 8

page 10

page 11

10/04/2021

Learning to Solve the AC Optimal Power Flow via a Lagrangian Approach

Using deep neural networks to predict the solutions of AC optimal power ...
12/11/2020

Theory-guided hard constraint projection (HCP): a knowledge-based data-driven scientific machine learning method

Machine learning models have been successfully used in many scientific a...
09/08/2020

Weak Form Theory-guided Neural Network (TgNN-wf) for Deep Learning of Subsurface Single and Two-phase Flow

Deep neural networks (DNNs) are widely used as surrogate models in geoph...
01/26/2020

A Lagrangian Dual Framework for Deep Neural Networks with Constraints

A variety of computationally challenging constrained optimization proble...
04/29/2022

AL-PINNs: Augmented Lagrangian relaxation method for Physics-Informed Neural Networks

Physics-Informed Neural Networks (PINNs) has become a prominent applicat...
05/11/2022

AutoKE: An automatic knowledge embedding framework for scientific machine learning

Imposing physical constraints on neural networks as a method of knowledg...
05/01/2020

A Dual-Dimer Method for Training Physics-Constrained Neural Networks with Minimax Architecture

Data sparsity is a common issue to train machine learning tools such as ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.