Distributed Control of Partial Differential Equations Using Convolutional Reinforcement Learning

01/25/2023
by   Sebastian Peitz, et al.
135

We present a convolutional framework which significantly reduces the complexity and thus, the computational effort for distributed reinforcement learning control of dynamical systems governed by partial differential equations (PDEs). Exploiting translational invariances, the high-dimensional distributed control problem can be transformed into a multi-agent control problem with many identical, uncoupled agents. Furthermore, using the fact that information is transported with finite velocity in many cases, the dimension of the agents' environment can be drastically reduced using a convolution operation over the state space of the PDE. In this setting, the complexity can be flexibly adjusted via the kernel width or by using a stride greater than one. Moreover, scaling from smaller to larger systems – or the transfer between different domains – becomes a straightforward task requiring little effort. We demonstrate the performance of the proposed framework using several PDE examples with increasing complexity, where stabilization is achieved by training a low-dimensional deep deterministic policy gradient agent using minimal computing resources.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/21/2021

Deep Reinforcement Learning for Online Control of Stochastic Partial Differential Equations

In many areas, such as the physical sciences, life sciences, and finance...
research
06/13/2018

Reinforcement Learning with Function-Valued Action Spaces for Partial Differential Equation Control

Recent work has shown that reinforcement learning (RL) is a promising ap...
research
06/21/2022

Finite Expression Method for Solving High-Dimensional Partial Differential Equations

Designing efficient and accurate numerical solvers for high-dimensional ...
research
02/14/2023

Learning a model is paramount for sample efficiency in reinforcement learning control of PDEs

The goal of this paper is to make a strong point for the usage of dynami...
research
02/22/2021

Actor-Critic Method for High Dimensional Static Hamilton–Jacobi–Bellman Partial Differential Equations based on Neural Networks

We propose a novel numerical method for high dimensional Hamilton–Jacobi...
research
08/08/2023

Online identification and control of PDEs via Reinforcement Learning methods

We focus on the control of unknown Partial Differential Equations (PDEs)...
research
01/30/2017

Expert Level control of Ramp Metering based on Multi-task Deep Reinforcement Learning

This article shows how the recent breakthroughs in Reinforcement Learnin...

Please sign up or login with your details

Forgot password? Click here to reset