Distributed Machine Learning for Computational Engineering using MPI

11/02/2020
by   Kailai Xu, et al.
0

We propose a framework for training neural networks that are coupled with partial differential equations (PDEs) in a parallel computing environment. Unlike most distributed computing frameworks for deep neural networks, our focus is to parallelize both numerical solvers and deep neural networks in forward and adjoint computations. Our parallel computing model views data communication as a node in the computational graph for numerical simulations. The advantage of our model is that data communication and computing are cleanly separated and thus provide better flexibility, modularity, and testability. We demonstrate using various large-scale problems that we can achieve substantial acceleration by using parallel solvers for PDEs in training deep neural networks that are coupled with PDEs.

READ FULL TEXT

page 10

page 14

page 16

page 17

research
09/08/2020

Large-scale Neural Solvers for Partial Differential Equations

Solving partial differential equations (PDE) is an indispensable part of...
research
06/28/2023

Training Deep Surrogate Models with Large Scale Online Learning

The spatiotemporal resolution of Partial Differential Equations (PDEs) p...
research
07/24/2020

Deep Generative Models that Solve PDEs: Distributed Computing for Training Large Data-Free Models

Recent progress in scientific machine learning (SciML) has opened up the...
research
08/18/2020

Benchmarking network fabrics for data distributed training of deep neural networks

Artificial Intelligence/Machine Learning applications require the traini...
research
05/16/2023

A Note on Dimensionality Reduction in Deep Neural Networks using Empirical Interpolation Method

Empirical interpolation method (EIM) is a well-known technique to effici...
research
05/10/2017

A performance spectrum for parallel computational frameworks that solve PDEs

Important computational physics problems are often large-scale in nature...
research
12/07/2021

A coarse space acceleration of deep-DDM

The use of deep learning methods for solving PDEs is a field in full exp...

Please sign up or login with your details

Forgot password? Click here to reset