Deep Structured Teams in Arbitrary-Size Linear Networks: Decentralized Estimation, Optimal Control and Separation Principle

by   Jalal Arabneydi, et al.

In this article, we introduce decentralized Kalman filters for linear quadratic deep structured teams. The agents in deep structured teams are coupled in dynamics, costs and measurements through a set of linear regressions of the states and actions (also called deep states and deep actions). The information structure is decentralized, where every agent observes a noisy measurement of its local state and the global deep state. Since the number of agents is often very large in deep structured teams, any naive approach to finding an optimal Kalman filter suffers from the curse of dimensionality. Moreover, due to the decentralized nature of information structure, the resultant optimization problem is non-convex, in general, where non-linear strategies can outperform linear ones. However, we prove that the optimal strategy is linear in the local state estimate as well as the deep state estimate and can be efficiently computed by two scale-free Riccati equations and Kalman filters. We propose a bi-level orthogonal approach across both space and time levels based on a gauge transformation technique to achieve the above result. We also establish a separation principle between optimal control and optimal estimation. Furthermore, we show that as the number of agents goes to infinity, the Kalman gain associated with the deep state estimate converges to zero at a rate inversely proportional to the number of agents. This leads to a fully decentralized approximate strategy where every agent predicts the deep state by its conditional and unconditional expected value, also known as the certainty equivalence approximation and (weighted) mean-field approximation, respectively.



page 1

page 2

page 3

page 4


Decentralized linear quadratic systems with major and minor agents and non-Gaussian noise

We consider a decentralized linear quadratic system with a major agent a...

Thompson sampling for linear quadratic mean-field teams

We consider optimal control of an unknown multi-agent linear quadratic (...

Reinforcement Learning in Deep Structured Teams: Initial Results with Finite and Infinite Valued Features

In this paper, we consider Markov chain and linear quadratic models for ...

Reinforcement Learning in Linear Quadratic Deep Structured Teams: Global Convergence of Policy Gradient Methods

In this paper, we study the global convergence of model-based and model-...

Approximate Optimal Filter for Linear Gaussian Time-invariant Systems

State estimation is critical to control systems, especially when the sta...

Adversarial Linear-Quadratic Mean-Field Games over Multigraphs

In this paper, we propose a game between an exogenous adversary and a ne...

Exact and Approximate Heterogeneous Bayesian Decentralized Data Fusion

In Bayesian peer-to-peer decentralized data fusion for static and dynami...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.