Backpropagation through Time and Space: Learning Numerical Methods with Multi-Agent Reinforcement Learning

03/16/2022
by   Elliot Way, et al.
0

We introduce Backpropagation Through Time and Space (BPTTS), a method for training a recurrent spatio-temporal neural network, that is used in a homogeneous multi-agent reinforcement learning (MARL) setting to learn numerical methods for hyperbolic conservation laws. We treat the numerical schemes underlying partial differential equations (PDEs) as a Partially Observable Markov Game (POMG) in Reinforcement Learning (RL). Similar to numerical solvers, our agent acts at each discrete location of a computational space for efficient and generalizable learning. To learn higher-order spatial methods by acting on local states, the agent must discern how its actions at a given spatiotemporal location affect the future evolution of the state. The manifestation of this non-stationarity is addressed by BPTTS, which allows for the flow of gradients across both space and time. The learned numerical policies are comparable to the SOTA numerics in two settings, the Burgers' Equation and the Euler Equations, and generalize well to other simulation set-ups.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/31/2022

Multi-Agent Learning of Numerical Methods for Hyperbolic PDEs with Factored Dec-MDP

Factored decentralized Markov decision process (Dec-MDP) is a framework ...
research
01/30/2017

Expert Level control of Ramp Metering based on Multi-task Deep Reinforcement Learning

This article shows how the recent breakthroughs in Reinforcement Learnin...
research
04/21/2022

Neural Networks with Local Converging Inputs (NNLCI) for Solving Conservation Laws, Part II: 2D Problems

In our prior work [arXiv:2109.09316], neural network methods with inputs...
research
11/02/2020

A scalable exponential-DG approach for nonlinear conservation laws: with application to Burger and Euler equations

We propose an Exponential DG approach for numerically solving partial di...
research
08/13/2018

On Passivity, Reinforcement Learning and Higher-Order Learning in Multi-Agent Finite Games

In this paper, we propose a passivity-based methodology for analysis and...
research
10/27/2021

Taylor Swift: Taylor Driven Temporal Modeling for Swift Future Frame Prediction

While recurrent neural networks (RNNs) demonstrate outstanding capabilit...
research
11/02/2022

Multi-Agent Reinforcement Learning for Adaptive Mesh Refinement

Adaptive mesh refinement (AMR) is necessary for efficient finite element...

Please sign up or login with your details

Forgot password? Click here to reset