Hierarchical Reinforcement Learning for Optimal Control of Linear Multi-Agent Systems: the Homogeneous Case

10/16/2020
by   Gangshan Jing, et al.
0

Individual agents in a multi-agent system (MAS) may have decoupled open-loop dynamics, but a cooperative control objective usually results in coupled closed-loop dynamics thereby making the control design computationally expensive. The computation time becomes even higher when a learning strategy such as reinforcement learning (RL) needs to be applied to deal with the situation when the agents dynamics are not known. To resolve this problem, this paper proposes a hierarchical RL scheme for a linear quadratic regulator (LQR) design in a continuous-time linear MAS. The idea is to exploit the structural properties of two graphs embedded in the Q and R weighting matrices in the LQR objective to define an orthogonal transformation that can convert the original LQR design to multiple decoupled smaller-sized LQR designs. We show that if the MAS is homogeneous then this decomposition retains closed-loop optimality. Conditions for decomposability, an algorithm for constructing the transformation matrix, a hierarchical RL algorithm, and robustness analysis when the design is applied to non-homogeneous MAS are presented. Simulations show that the proposed approach can guarantee significant speed-up in learning without any loss in the cumulative value of the LQR cost.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

04/29/2020

Reduced-Dimensional Reinforcement Learning Control using Singular Perturbation Approximations

We present a set of model-free, reduced-dimensional reinforcement learni...
08/14/2020

Model-Free Optimal Control of Linear Multi-Agent Systems via Decomposition and Hierarchical Approximation

Designing the optimal linear quadratic regulator (LQR) for a large-scale...
05/25/2021

KnowSR: Knowledge Sharing among Homogeneous Agents in Multi-agent Reinforcement Learning

Recently, deep reinforcement learning (RL) algorithms have made great pr...
08/31/2017

Optimal Distributed Control of Multi-agent Systems in Contested Environments via Reinforcement Learning

This paper presents a model-free reinforcement learning (RL) based distr...
02/01/2021

Hybrid Information-driven Multi-agent Reinforcement Learning

Information theoretic sensor management approaches are an ideal solution...
11/02/2020

Reinforcement Learning of Structured Control for Linear Systems with Unknown State Matrix

This paper delves into designing stabilizing feedback control gains for ...
04/11/2016

A statistical learning strategy for closed-loop control of fluid flows

This work discusses a closed-loop control strategy for complex systems u...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.