Distributed Algorithms for Linearly-Solvable Optimal Control in Networked Multi-Agent Systems

by   Neng Wan, et al.

Distributed algorithms for both discrete-time and continuous-time linearly solvable optimal control (LSOC) problems of networked multi-agent systems (MASs) are investigated in this paper. A distributed framework is proposed to partition the optimal control problem of a networked MAS into several local optimal control problems in factorial subsystems, such that each (central) agent behaves optimally to minimize the joint cost function of a subsystem that comprises a central agent and its neighboring agents, and the local control actions (policies) only rely on the knowledge of local observations. Under this framework, we not only preserve the correlations between neighboring agents, but moderate the communication and computational complexities by decentralizing the sampling and computational processes over the network. For discrete-time systems modeled by Markov decision processes, the joint Bellman equation of each subsystem is transformed into a system of linear equations and solved using parallel programming. For continuous-time systems modeled by Itô diffusion processes, the joint optimality equation of each subsystem is converted into a linear partial differential equation, whose solution is approximated by a path integral formulation and a sample-efficient relative entropy policy search algorithm, respectively. The learned control policies are generalized to solve the unlearned tasks by resorting to the compositionality principle, and illustrative examples of cooperative UAV teams are provided to verify the effectiveness and advantages of these algorithms.


page 1

page 2

page 3

page 4


Cooperative Path Integral Control for Stochastic Multi-Agent Systems

A distributed stochastic optimal control solution is presented for coope...

Compositionality of Linearly Solvable Optimal Control in Networked Multi-Agent Systems

In this paper, we discuss the methodology of generalizing the optimal co...

Distributed Dynamic Programming forNetworked Multi-Agent Markov Decision Processes

The main goal of this paper is to investigate distributed dynamic progra...

Variational Principles for Optimal Control of Left-Invariant Multi-Agent Systems with Asymmetric Formation Constraints

We study an optimal control problem for a multi-agent system modeled by ...

Thompson Sampling Efficiently Learns to Control Diffusion Processes

Diffusion processes that evolve according to linear stochastic different...

Temporal Parallelisation of the HJB Equation and Continuous-Time Linear Quadratic Control

This paper presents a mathematical formulation to perform temporal paral...

Chance-Constrained Stochastic Optimal Control via Path Integral and Finite Difference Methods

This paper addresses a continuous-time continuous-space chance-constrain...

Please sign up or login with your details

Forgot password? Click here to reset