Fully differentiable model discovery

06/09/2021
by   Gert-Jan Both, et al.
16

Model discovery aims at autonomously discovering differential equations underlying a dataset. Approaches based on Physics Informed Neural Networks (PINNs) have shown great promise, but a fully-differentiable model which explicitly learns the equation has remained elusive. In this paper we propose such an approach by combining neural network based surrogates with Sparse Bayesian Learning (SBL). We start by reinterpreting PINNs as multitask models, applying multitask learning using uncertainty, and show that this leads to a natural framework for including Bayesian regression techniques. We then construct a robust model discovery algorithm by using SBL, which we showcase on various datasets. Concurrently, the multitask approach allows the use of probabilistic approximators, and we show a proof of concept using normalizing flows to directly learn a density model from single particle data. Our work expands PINNs to various types of neural network architectures, and connects neural network-based surrogates to the rich field of Bayesian parameter inference.

READ FULL TEXT

Authors

page 6

page 9

06/28/2016

Modeling Industrial ADMET Data with Multitask Networks

Deep learning methods such as multitask neural networks have recently be...
01/12/2021

Bayesian equation selection on sparse data for discovery of stochastic dynamical systems

Often the underlying system of differential equations driving a stochast...
05/31/2021

Robust discovery of partial differential equations in complex situations

Data-driven discovery of partial differential equations (PDEs) has achie...
02/06/2015

Massively Multitask Networks for Drug Discovery

Massively multitask neural architectures provide a learning framework fo...
02/12/2019

Multitask Learning for Polyphonic Piano Transcription, a Case Study

Viewing polyphonic piano transcription as a multitask learning problem, ...
07/03/2020

Differentiable Causal Discovery from Interventional Data

Discovering causal relationships in data is a challenging task that invo...
11/09/2020

Sparsely constrained neural networks for model discovery of PDEs

Sparse regression on a library of candidate features has developed as th...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.