
Multifidelity classification using Gaussian processes: accelerating the prediction of largescale computational models
Machine learning techniques typically rely on large datasets to create accurate classifiers. However, there are situations when data is scarce and expensive to acquire. This is the case of studies that rely on stateoftheart computational models which typically take days to run, thus hindering the potential of machine learning tools. In this work, we present a novel classifier that takes advantage of lower fidelity models and inexpensive approximations to predict the binary output of expensive computer simulations. We postulate an autoregressive model between the different levels of fidelity with Gaussian process priors. We adopt a fully Bayesian treatment for the hyperparameters and use Markov Chain Mont Carlo samplers. We take advantage of the probabilistic nature of the classifier to implement active learning strategies. We also introduce a sparse approximation to enhance the ability of themultifidelity classifier to handle large datasets. We test these multifidelity classifiers against their singlefidelity counterpart with synthetic data, showing a median computational cost reduction of 23 target accuracy of 90 multifidelity classifier achieves an F1 score, the harmonic mean of precision and recall, of 99.6 both are trained with 50 samples. In general, our results show that the multifidelity classifiers outperform their singlefidelity counterpart in terms of accuracy in all cases. We envision that this new tool will enable researchers to study classification problems that would otherwise be prohibitively expensive. Source code is available at https://github.com/fsahli/MFclass.
05/09/2019 ∙ by Francisco Sahli Costabal, et al. ∙ 10 ∙ shareread it

Adversarial Uncertainty Quantification in PhysicsInformed Neural Networks
We present a deep learning framework for quantifying and propagating uncertainty in systems governed by nonlinear differential equations using physicsinformed neural networks. Specifically, we employ latent variable models to construct probabilistic representations for the system states, and put forth an adversarial inference procedure for training them on data, while constraining their predictions to satisfy given physical laws expressed by partial differential equations. Such physicsinformed constraints provide a regularization mechanism for effectively training deep generative models as surrogates of physical systems in which the cost of data acquisition is high, and training datasets are typically small. This provides a flexible framework for characterizing uncertainty in the outputs of physical systems due to randomness in their inputs or noise in their observations that entirely bypasses the need for repeatedly sampling expensive experiments or numerical simulators. We demonstrate the effectiveness of our approach through a series of examples involving uncertainty propagation in nonlinear conservation laws, and the discovery of constitutive laws for flow through porous media directly from noisy data.
11/09/2018 ∙ by Yibo Yang, et al. ∙ 6 ∙ shareread it

Physics Informed Deep Learning (Part II): Datadriven Discovery of Nonlinear Partial Differential Equations
We introduce physics informed neural networks  neural networks that are trained to solve supervised learning tasks while respecting any given law of physics described by general nonlinear partial differential equations. In this second part of our twopart treatise, we focus on the problem of datadriven discovery of partial differential equations. Depending on whether the available data is scattered in spacetime or arranged in fixed temporal snapshots, we introduce two main classes of algorithms, namely continuous time and discrete time models. The effectiveness of our approach is demonstrated using a wide range of benchmark problems in mathematical physics, including conservation laws, incompressible fluid flow, and the propagation of nonlinear shallowwater waves.
11/28/2017 ∙ by Maziar Raissi, et al. ∙ 0 ∙ shareread it

Multistep Neural Networks for Datadriven Discovery of Nonlinear Dynamical Systems
The process of transforming observed data into predictive mathematical models of the physical world has always been paramount in science and engineering. Although data is currently being collected at an everincreasing pace, devising meaningful models out of such observations in an automated fashion still remains an open problem. In this work, we put forth a machine learning approach for identifying nonlinear dynamical systems from data. Specifically, we blend classical tools from numerical analysis, namely the multistep timestepping schemes, with powerful nonlinear function approximators, namely deep neural networks, to distill the mechanisms that govern the evolution of a given dataset. We test the effectiveness of our approach for several benchmark problems involving the identification of complex, nonlinear and chaotic dynamics, and we demonstrate how this allows us to accurately learn the dynamics, forecast future states, and identify basins of attraction. In particular, we study the Lorenz system, the fluid flow behind a cylinder, the Hopf bifurcation, and the Glycoltic oscillator model as an example of complicated nonlinear dynamics typical of biological systems.
01/04/2018 ∙ by Maziar Raissi, et al. ∙ 0 ∙ shareread it

Numerical Gaussian Processes for Timedependent and Nonlinear Partial Differential Equations
We introduce the concept of numerical Gaussian processes, which we define as Gaussian processes with covariance functions resulting from temporal discretization of timedependent partial differential equations. Numerical Gaussian processes, by construction, are designed to deal with cases where: (1) all we observe are noisy data on blackbox initial conditions, and (2) we are interested in quantifying the uncertainty associated with such noisy data in our solutions to timedependent partial differential equations. Our method circumvents the need for spatial discretization of the differential operators by proper placement of Gaussian process priors. This is an attempt to construct structured and dataefficient learning machines, which are explicitly informed by the underlying physics that possibly generated the observed data. The effectiveness of the proposed approach is demonstrated through several benchmark problems involving linear and nonlinear timedependent operators. In all examples, we are able to recover accurate approximations of the latent solutions, and consistently propagate uncertainty, even in cases involving very long time integration.
03/29/2017 ∙ by Maziar Raissi, et al. ∙ 0 ∙ shareread it

Machine Learning of SpaceFractional Differential Equations
Datadriven discovery of "hidden physics"  i.e., machine learning of differential equation models underlying observed data  has recently been approached by embedding the discovery problem into a Gaussian Process regression of spatial data, treating and discovering unknown equation parameters as hyperparameters of a modified "physics informed" Gaussian Process kernel. This kernel includes the parametrized differential operators applied to a prior covariance kernel. We extend this framework to linear spacefractional differential equations. The methodology is compatible with a wide variety of fractional operators in R^d and stationary covariance kernels, including the Matern class, and can optimize the Matern parameter during training. We provide a userfriendly and feasible way to perform fractional derivatives of kernels, via a unified set of ddimensional Fourier integral formulas amenable to generalized GaussLaguerre quadrature. The implementation of fractional derivatives has several benefits. First, it allows for discovering fractionalorder PDEs for systems characterized by heavy tails or anomalous diffusion, bypassing the analytical difficulty of fractional calculus. Data sets exhibiting such features are of increasing prevalence in physical and financial domains. Second, a single fractionalorder archetype allows for a derivative of arbitrary order to be learned, with the order itself being a parameter in the regression. This is advantageous even when used for discovering integerorder equations; the user is not required to assume a "dictionary" of derivatives of various orders, and directly controls the parsimony of the models being discovered. We illustrate on several examples, including fractionalorder interpolation of advectiondiffusion and modeling relative stock performance in the S&P 500 with alphastable motion via a fractional diffusion equation.
08/02/2018 ∙ by Mamikon Gulian, et al. ∙ 0 ∙ shareread it

Physics Informed Deep Learning (Part I): Datadriven Solutions of Nonlinear Partial Differential Equations
We introduce physics informed neural networks  neural networks that are trained to solve supervised learning tasks while respecting any given law of physics described by general nonlinear partial differential equations. In this two part treatise, we present our developments in the context of solving two main classes of problems: datadriven solution and datadriven discovery of partial differential equations. Depending on the nature and arrangement of the available data, we devise two distinct classes of algorithms, namely continuous time and discrete time models. The resulting neural networks form a new class of dataefficient universal function approximators that naturally encode any underlying physical laws as prior information. In this first part, we demonstrate how these networks can be used to infer solutions to partial differential equations, and obtain physicsinformed surrogate models that are fully differentiable with respect to all input coordinates and free parameters.
11/28/2017 ∙ by Maziar Raissi, et al. ∙ 0 ∙ shareread it

Physicsinformed deep generative models
We consider the application of deep generative models in propagating uncertainty through complex physical systems. Specifically, we put forth an implicit variational inference formulation that constrains the generative model output to satisfy given physical laws expressed by partial differential equations. Such physicsinformed constraints provide a regularization mechanism for effectively training deep probabilistic models for modeling physical systems in which the cost of data acquisition is high and training datasets are typically small. This provides a scalable framework for characterizing uncertainty in the outputs of physical systems due to randomness in their inputs or noise in their observations. We demonstrate the effectiveness of our approach through a canonical example in transport dynamics.
12/09/2018 ∙ by Yibo Yang, et al. ∙ 0 ∙ shareread it

Conditional deep surrogate models for stochastic, highdimensional, and multifidelity systems
We present a probabilistic deep learning methodology that enables the construction of predictive datadriven surrogates for stochastic systems. Leveraging recent advances in variational inference with implicit distributions, we put forth a statistical inference framework that enables the endtoend training of surrogate models on paired inputoutput observations that may be stochastic in nature, originate from different information sources of variable fidelity, or be corrupted by complex noise processes. The resulting surrogates can accommodate highdimensional inputs and outputs and are able to return predictions with quantified uncertainty. The effectiveness our approach is demonstrated through a series of canonical studies, including the regression of noisy data, multifidelity modeling of stochastic processes, and uncertainty propagation in highdimensional dynamical systems.
01/15/2019 ∙ by Yibo Yang, et al. ∙ 0 ∙ shareread it

PhysicsConstrained Deep Learning for Highdimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
Surrogate modeling and uncertainty quantification tasks for PDE systems are most often considered as supervised learning problems where input and output data pairs are used for training. The construction of such emulators is by definition a small data problem which poses challenges to deep learning approaches that have been developed to operate in the big data regime. Even in cases where such models have been shown to have good predictive capability in high dimensions, they fail to address constraints in the data implied by the PDE model. This paper provides a methodology that incorporates the governing equations of the physical model in the loss/likelihood functions. The resulting physicsconstrained, deep learning models are trained without any labeled data (e.g. employing only input data) and provide comparable predictive responses with datadriven models while obeying the constraints of the problem at hand. This work employs a convolutional encoderdecoder neural network approach as well as a conditional flowbased generative model for the solution of PDEs, surrogate model construction, and uncertainty quantification tasks. The methodology is posed as a minimization problem of the reverse KullbackLeibler (KL) divergence between the model predictive density and the reference conditional density, where the later is defined as the BoltzmannGibbs distribution at a given inverse temperature with the underlying potential relating to the PDE system of interest. The generalization capability of these models to outofdistribution input is considered. Quantification and interpretation of the predictive uncertainty is provided for a number of problems.
01/18/2019 ∙ by Yinhao Zhu, et al. ∙ 0 ∙ shareread it

Machine learning in cardiovascular flows modeling: Predicting pulse wave propagation from noninvasive clinical measurements using physicsinformed deep learning
Advances in computational science offer a principled pipeline for predictive modeling of cardiovascular flows and aspire to provide a valuable tool for monitoring, diagnostics and surgical planning. Such models can be nowadays deployed on large patientspecific topologies of systemic arterial networks and return detailed predictions on flow patterns, wall shear stresses, and pulse wave propagation. However, their success heavily relies on tedious preprocessing and calibration procedures that typically induce a significant computational cost, thus hampering their clinical applicability. In this work we put forth a machine learning framework that enables the seamless synthesis of noninvasive invivo measurement techniques and computational flow dynamics models derived from first physical principles. We illustrate this new paradigm by showing how onedimensional models of pulsatile flow can be used to constrain the output of deep neural networks such that their predictions satisfy the conservation of mass and momentum principles. Once trained on noisy and scattered clinical data of flow and wall displacement, these networks can return physically consistent predictions for velocity, pressure and wall displacement pulse wave propagation, all without the need to employ conventional simulators. A simple postprocessing of these outputs can also provide a cheap and effective way for estimating Windkessel model parameters that are required for the calibration of traditional computational models. The effectiveness of the proposed techniques is demonstrated through a series of prototype benchmarks, as well as a realistic clinical case involving invivo measurements near the aorta/carotid bifurcation of a healthy human subject.
05/13/2019 ∙ by Georgios Kissas, et al. ∙ 0 ∙ shareread it
Paris Perdikaris
is this you? claim profile