Fully differentiable model discovery

by   Gert-Jan Both, et al.

Model discovery aims at autonomously discovering differential equations underlying a dataset. Approaches based on Physics Informed Neural Networks (PINNs) have shown great promise, but a fully-differentiable model which explicitly learns the equation has remained elusive. In this paper we propose such an approach by combining neural network based surrogates with Sparse Bayesian Learning (SBL). We start by reinterpreting PINNs as multitask models, applying multitask learning using uncertainty, and show that this leads to a natural framework for including Bayesian regression techniques. We then construct a robust model discovery algorithm by using SBL, which we showcase on various datasets. Concurrently, the multitask approach allows the use of probabilistic approximators, and we show a proof of concept using normalizing flows to directly learn a density model from single particle data. Our work expands PINNs to various types of neural network architectures, and connects neural network-based surrogates to the rich field of Bayesian parameter inference.



page 6

page 9


Modeling Industrial ADMET Data with Multitask Networks

Deep learning methods such as multitask neural networks have recently be...

Bayesian equation selection on sparse data for discovery of stochastic dynamical systems

Often the underlying system of differential equations driving a stochast...

Robust discovery of partial differential equations in complex situations

Data-driven discovery of partial differential equations (PDEs) has achie...

Massively Multitask Networks for Drug Discovery

Massively multitask neural architectures provide a learning framework fo...

Multitask Learning for Polyphonic Piano Transcription, a Case Study

Viewing polyphonic piano transcription as a multitask learning problem, ...

Differentiable Causal Discovery from Interventional Data

Discovering causal relationships in data is a challenging task that invo...

Sparsely constrained neural networks for model discovery of PDEs

Sparse regression on a library of candidate features has developed as th...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.