Decentralized linear quadratic systems with major and minor agents and non-Gaussian noise

04/24/2020
by   Mohammad Afshari, et al.
0

We consider a decentralized linear quadratic system with a major agent and a collection of minor agents. The agents are coupled in their dynamics as well as a quadratic cost. In particular, the dynamics are linear; the state and control action of the major agent affect the state evolution of all the minor agents but the state and the control action of the minor agents do not affect the state evolution of the major or other minor agents. The system has partial output feedback with partially nested information structure. In particular, the major agent perfectly observes its own state while each minor agent perfectly observes the state of the major agent and partially observes its own state. It is not assumed that the noise process has a Gaussian distribution. For this model, we characterize the structure of the optimal and the best linear strategies. We show that the optimal control of the major agent is a linear function of the major agent's MMSE (minimum mean squared error) estimate of the system state and the optimal control of a minor agent is a linear function of the major agent's MMSE estimate of the system state and a "correction term" which depends on the difference of the minor agent's MMSE estimate of its local state and the major agent's MMSE estimate of the minor agent's local state. The major agent's MMSE estimate is a linear function of its observations while the minor agent's MMSE estimate is a non-linear function of its observations which is updated according to the non-linear Bayesian filter. We show that if we replace the minor agent's MMSE estimate by its LLMS (linear least mean square) estimate, then the resultant strategy is the best linear control strategy. We prove the result using a direct proof which is based on conditional independence, splitting of the state and control actions, simplifying the per-step cost, orthogonality principle, and completion of squares.

READ FULL TEXT
10/23/2021

Deep Structured Teams in Arbitrary-Size Linear Networks: Decentralized Estimation, Optimal Control and Separation Principle

In this article, we introduce decentralized Kalman filters for linear qu...
11/09/2020

Thompson sampling for linear quadratic mean-field teams

We consider optimal control of an unknown multi-agent linear quadratic (...
09/30/2020

Cooperative Path Integral Control for Stochastic Multi-Agent Systems

A distributed stochastic optimal control solution is presented for coope...
05/15/2021

Regret Analysis of Distributed Online LQR Control for Unknown LTI Systems

Online learning has recently opened avenues for rethinking classical opt...
08/09/2022

Mathematical Foundations of Complex Tonality

Equal temperament, in which semitones are tuned in the irrational ratio ...
07/28/2020

Team Deep Mixture of Experts for Distributed Power Control

In the context of wireless networking, it was recently shown that multip...
01/16/2019

ReNeg and Backseat Driver: Learning from Demonstration with Continuous Human Feedback

In autonomous vehicle (AV) control, allowing mistakes can be quite dange...