Decentralized linear quadratic systems with major and minor agents and non-Gaussian noise
share
We consider a decentralized linear quadratic system with a major agent and a collection of minor agents. The agents are coupled in their dynamics as well as a quadratic cost. In particular, the dynamics are linear; the state and control action of the major agent affect the state evolution of all the minor agents but the state and the control action of the minor agents do not affect the state evolution of the major or other minor agents. The system has partial output feedback with partially nested information structure. In particular, the major agent perfectly observes its own state while each minor agent perfectly observes the state of the major agent and partially observes its own state. It is not assumed that the noise process has a Gaussian distribution. For this model, we characterize the structure of the optimal and the best linear strategies. We show that the optimal control of the major agent is a linear function of the major agent's MMSE (minimum mean squared error) estimate of the system state and the optimal control of a minor agent is a linear function of the major agent's MMSE estimate of the system state and a "correction term" which depends on the difference of the minor agent's MMSE estimate of its local state and the major agent's MMSE estimate of the minor agent's local state. The major agent's MMSE estimate is a linear function of its observations while the minor agent's MMSE estimate is a non-linear function of its observations which is updated according to the non-linear Bayesian filter. We show that if we replace the minor agent's MMSE estimate by its LLMS (linear least mean square) estimate, then the resultant strategy is the best linear control strategy. We prove the result using a direct proof which is based on conditional independence, splitting of the state and control actions, simplifying the per-step cost, orthogonality principle, and completion of squares.
READ FULL TEXT