Differential radial basis function network for sequence modelling

10/13/2020
by   Kojo Sarfo Gyamfi, et al.
0

We propose a differential radial basis function (RBF) network termed RBF-DiffNet – whose hidden layer blocks are partial differential equations (PDEs) linear in terms of the RBF – to make the baseline RBF network robust to noise in sequential data. Assuming that the sequential data derives from the discretisation of the solution to an underlying PDE, the differential RBF network learns constant linear coefficients of the PDE, consequently regularising the RBF network by following modified backward-Euler updates. We experimentally validate the differential RBF network on the logistic map chaotic timeseries as well as on 30 real-world timeseries provided by Walmart in the M5 forecasting competition. The proposed model is compared with the normalised and unnormalised RBF networks, ARIMA, and ensembles of multilayer perceptrons (MLPs) and recurrent networks with long short-term memory (LSTM) blocks. From the experimental results, RBF-DiffNet consistently shows a marked reduction over the baseline RBF network in terms of the prediction error (e.g., 26 RBF-DiffNet also shows a comparable performance to the LSTM ensemble at less than one-sixteenth the LSTM computational time. Our proposed network consequently enables more accurate predictions – in the presence of observational noise – in sequence modelling tasks such as timeseries forecasting that leverage the model interpretability, fast training, and function approximation properties of the RBF network.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

10/22/2019

Learning Partial Differential Equations from Data Using Neural Networks

We develop a framework for estimating unknown partial differential equat...
03/14/2020

Error bounds for PDE-regularized learning

In this work we consider the regularization of a supervised learning pro...
02/04/2021

Machine Learning for Auxiliary Sources

We rewrite the numerical ansatz of the Method of Auxiliary Sources (MAS)...
11/30/2018

PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network

Partial differential equations (PDEs) are commonly derived based on empi...
04/25/2015

Differential Recurrent Neural Networks for Action Recognition

The long short-term memory (LSTM) neural network is capable of processin...
08/23/2021

Deep learning for surrogate modelling of 2D mantle convection

Traditionally, 1D models based on scaling laws have been used to paramet...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.