MC-LSTM: Mass-Conserving LSTM

01/13/2021
by   Pieter-Jan Hoedt, et al.
13

The success of Convolutional Neural Networks (CNNs) in computer vision is mainly driven by their strong inductive bias, which is strong enough to allow CNNs to solve vision-related tasks with random weights, meaning without learning. Similarly, Long Short-Term Memory (LSTM) has a strong inductive bias towards storing information over time. However, many real-world systems are governed by conservation laws, which lead to the redistribution of particular quantities – e.g. in physical and economical systems. Our novel Mass-Conserving LSTM (MC-LSTM) adheres to these conservation laws by extending the inductive bias of LSTM to model the redistribution of those stored quantities. MC-LSTMs set a new state-of-the-art for neural arithmetic units at learning arithmetic operations, such as addition tasks, which have a strong conservation law, as the sum is constant over time. Further, MC-LSTM is applied to traffic forecasting, modelling a pendulum, and a large benchmark dataset in hydrology, where it sets a new state-of-the-art for predicting peak flows. In the hydrology example, we show that MC-LSTM states correlate with real-world processes and are therefore interpretable.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 8

03/31/2017

Factorization tricks for LSTM networks

We present two simple ways of reducing the number of parameters and acce...
01/23/2020

Low-Complexity LSTM Training and Inference with FloatSD8 Weight Representation

The FloatSD technology has been shown to have excellent performance on l...
09/18/2021

PCNN: A physics-constrained neural network for multiphase flows

The present study develops a physics-constrained neural network (PCNN) t...
05/05/2020

Long short-term memory networks and laglasso for bond yield forecasting: Peeping inside the black box

Modern decision-making in fixed income asset management benefits from in...
07/13/2021

Fast-Slow Streamflow Model Using Mass-Conserving LSTM

Streamflow forecasting is key to effectively managing water resources an...
07/16/2018

Longitudinal detection of radiological abnormalities with time-modulated LSTM

Convolutional neural networks (CNNs) have been successfully employed in ...
03/24/2018

Multi-range Reasoning for Machine Comprehension

We propose MRU (Multi-Range Reasoning Units), a new fast compositional e...

Code Repositories

mc-lstm

Experiments with Mass Conserving LSTMs


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.