Sensors are often geographically deployed to collect measurements over large-scale networked dynamical systems, which are used by state estimators (implementing state observers) to retrieve an estimate of the overall state of the system. Then, the estimate is provided to the actuator that implements a controller to steer the dynamical system to the desired state [1, 2, 3, 4]. Estimators can be full-state, reduced and extended state observers that implicitly explore the trade-offs between communication and estimation [5, 6]. In the context of distributed estimation, additional information used by the state estimators is shared to improve the quality of the estimate. For example, they can share either the estimate, the error between predicted state observation and the measurement, or the innovation used as part of the state estimation process [7, 8, 4, 9]. The information is shared by resorting to communication between different sensors’ computational units, and the communication capabilities impact the ability to retrieve the estimate of the state. Therefore, it is fundamental to understand which communication is required to ensure a successful recovery of the system state [10, 11].
Most of the observers implemented in large-scale systems require a large amount of information being exchanged through communication (i.e., the state, the error between measurement and predicted state observation). Furthermore, the estimators are shown to be asymptotically stable (not always with an arbitrary error decay) which might restrict the actuation performance in the context of large-scale networked dynamical systems. To overcome such limitations, in , we proposed an approach that equips the sensors with scalar states which are exchanged with other sensors, and together with sensors measurements, suffice to retrieve in finite-time the state of the networked dynamical system and those of the sensors, which we refer to as limited communication decentralized estimation scheme. Subsequently, the information being exchanged between sensors is reduced to the bare minimum, and the communication topologies analyzed are designed to ensure sensor-network state recovery.
In this paper, we seek to better understand the restrictions and trade-offs of the limited communication decentralized estimation scheme. Specifically, the main contributions of this paper are: (i) we waive some implicit assumptions made in  about the communication scheme performed by the sensors (which are in general only sufficient, as we emphasize in Remark 2); (ii) we explore the implications in two different setups: (a) the sensors are memoryless (i.e., they do not keep track of their previous state); and (b) sensors might not have the capacity to discern the contributions and/or state of other sensors (e.g., those relying on radio technology); and (iii) we leverage these new conditions to cast the problem of determining the minimum communication cost required to deploy a limited communication decentralized estimation scheme as an integer programming problem.
Ii Problem Statement
Let the evolution of a (possibly) large-scale networked dynamical system be captured by
where is the state of the system. Consider sensors with measurements described as follows:
is the output vector describing the contributions of the different observed state variables. We assume thatis observable, but not necessarily observable from a specific sensor , i.e., is not necessarily observable.
In the limited communication decentralized estimation scheme, we consider that the sensors possess a scalar state and can communicate with each other. During this process, they share their states, which enables the retrieval of the state of both the networked dynamical system and the sensors. The communication capabilities are captured by a directed communication graph , where the set of vertices labels the sensors, and an edge translates in the capability of sensor to receive data from sensor . Besides, each sensor computes a linear combination of the (scalar) measurement and the scalar data received from the neighboring sensors, i.e., , which can be described as follows:
where is the augmented system’s state and the dynamics between sensors induced by the communication graph, i.e., when and zero otherwise. It is worth noticing that some of the weights may be set to zero, and, in particular, if for then we are dealing with memoryless sensors that work as relays – which cannot be addressed by the setup explored in . Additionally, the augmented system’s output is described as follows:
where is the sub-matrix containing the rows of the identity matrix with indices in . In particular, when the linear combination of incoming sensor’s states is performed (locally) at sensor , or when sensors do not have the capacity to discern the contributions and/or state of other sensors (e.g., those relying on radio technology). The latter case cannot be addressed by the setup explored in .
In this paper, we seek solutions to the following problems:
Problem 1: Characterize the necessary and sufficient conditions that must be satisfied by (and, subsequently, by ) ensuring that is observable.
In particular, we provide the characterization required in the memoryless sensor scenario, and in the case where a sensor only has access to its own state. Next, we propose to determine communication topologies that ensure the necessary and sufficient conditions required to solve the previous problem, while minimizing the communication cost between the different sensors:
Problem 2: Let be the communication cost incurred by establishing a communication link between the sensors to obtain a communication graph . We aim to determine that solves the following optimization problem:
Iii Terminology and Previous Results
In what follows, we rely on structural systems theory  to assess system theoretical properties by considering only the inter-dependencies between states and sensors. One such system property is that of structural observability that considers the sparsity binary patterns , where an entry in these matrices is zero if there is no direct dependency between two (state or sensor) variables and one otherwise [14, 15]. A pair is structurally observable if there exists an observable pair such that the zero entries in are also zero in . Subsequently, it can be proved that if such an observable pair exists, then almost all possible pairs satisfying the sparsity pattern are also observable. Furthermore, structural properties (e.g., structural observability) are necessary to ensure non-structural properties (observability). Therefore, in Section V, we rely on structural systems to ensure first the necessary conditions, and then we show that in fact these are also sufficient.
One of the key features of structural systems theory is that we can interpret the sparsity patterns as a directed state graph and state-output graph , where the vertices are labeled by the states and sensors and the edges capture the inter-dependencies between state and sensor variables as follows: and . We will use for brevity. Additionally, we can use graph-theoretical notions, e.g., paths and cycles, to address the structural properties. In particular, to characterize structural observability, we introduce the notion of bipartite graph associated with the state graph and state-output graph. The state bipartite graph (resp., the state-output bipartite graph ), consists of two sets (resp., and ) that can be graphically interpreted and to which we refer to as left and right set of vertices. Edges between the left and right set of vertices encode the dependencies described by the edge-set of the directed state graph (resp., directed state-output graph). Also, due to the correspondence between these edges, paths and cycles in the state-output graph can be captured by subsets of vertex-disjoint edges in the state and state-output bipartite graph, which are referred to as matchings, and the subset with the largest number of edges referred to as maximum matching. Consequently, those left (resp., right) vertices in the state and state-output bipartite graph that do not belong to any edge in the matching are referred to as left-unmatched (resp., right-unmatched) vertices. Accordingly, we have the following result:
Lemma 1 ()
Consider the digraph and let be a maximum matching associated to the state-output bipartite graph . Then, the digraph comprises a disjoint union of cycles and elementary paths, from the left-unmatched vertices to the right-unmatched vertices of , that span . Moreover, such a decomposition is minimal, in the sense that no other spanning subgraph decomposition of into elementary paths and cycles contains strictly fewer elementary paths.
The different graph-theoretic concepts can come together to assess structural observability of as follows.
Theorem 1 ()
Let denote the state-output digraph and the state-output bipartite representation. The pair is structurally observable if and only if the following two conditions hold:
there is a path from every state vertex to an output vertex in ; and
there exists a maximum matching associated to such that the left-unmatched vertices .
Therefore, as previously mentioned, we can build upon these results to analyze and design the limited communication decentralized estimation scheme. In , we have provided necessary and sufficient conditions that needs to satisfy to ensure observability of under the following simplifying implicit assumption.
In other words,  excludes the case of memoryless sensors, which we address in this paper. We also explore other setups, e.g., when the sensors are not able to differentiate the individual contributions of other sensors due to the technology used. Next, we state two of the main results in  for ease of comparison with the main results attained in this work, where we waive the implicit assumption stated above.
Theorem 2 ()
Let be the state digraph, where corresponds to the labels of the state vertices and to the labels of the sensors’ states. In addition, let be the set of in-neighbors of a vertex representing a sensor in , . The following two conditions are necessary and sufficient to ensure that , for , is generically observable:
for every there must exist a directed path from any ;
for every there must exist a set of left-unmatched vertices , associated with a maximum matching of the bipartite representation of , such that and .
Hence, Theorem 2 can be used to obtain the next result to Problem 1 under the implicit assumption stated above.
Theorem 3 ()
If is observable and is structurally observable , then almost all realizations of ensure that is observable.
For brevity’s sake, we will use the shortened notation in the rest of the paper.
Iv Limited Communication Analysis and Design
In this section, we introduce the main results of this paper. Lemma 2 shows that the sensor capabilities impose strong constraints on the network’s structure required to ensure structural observability. This technical result plays a key role in understanding Theorem 4, which states the necessary and sufficient conditions required to address Problem 1. Specifically, it provides the conditions for the communication graph such that is observable, . Next, we consider the design of communication graphs that attain the former conditions, while minimizing the total cost incurred by the communication between the sensors (Problem 2). In particular, we cast the problem as an integer programming problem that can be solved with off-the-shelf solvers.
We start by showing that the structural observability of a pair , that is often assessed through the state-output graph properties (as captured in Theorem 1), can enforce a particular structure of under certain sensing capabilities.
Let be the canonical column-vector with one in the ’th position and the remaining entries equal to zero. Given a structured adjacency matrix of a graph, , the pairs , for , are structurally observable if and only if the associated state digraph is strongly connected and spanned by a disjoint union of cycles.
(Necessity) Assume is structurally observable, for all . Suppose by contradiction that is not strongly connected. If is not strongly connected, then the state-output digraph is also not strongly connected and its directed acyclic representation contains a number of strongly connected components. Since the output vertex in is one vertex connected only to vertex , then it readily follows that condition (i) of Theorem 1 cannot hold.
Next, we prove that is spanned by a disjoint union of cycles. Consider condition (ii) of Theorem 1: there exists a maximum matching in such that there is no left-unmatched vertex in . There are two possibilities for the maximum matchings in : (a) and (b) . Case (a) means that there is a perfect matching in , i.e., from Lemma 1, is spanned by a disjoint union of cycles.
We want to prove now that case (b) cannot happen when is structurally observable. Let us first address the case when . Let represent the set of vertices in the graph described by . Consider a maximum matching in that has and , for some vertices . Since is strongly connected, then, there exists a neighbor of such that . Let be another maximum matching in such that and , for some vertex . By Lemma 4 in , there exists a maximum matching such that and . However, is also a maximum matching, in fact, a perfect matching, which leads to a contradiction of the fact that is a maximum matching. Therefore, the set of left-unmatched vertices has to be empty, meaning a maximum matching is also a perfect matching, leading to the fact that is spanned by a disjoint union of cycles. Now, for the case when , we can iteratively find augmented paths  and construct larger cardinality maximum matchings, while thus reducing the cardinality of the set of left-unmatched vertices with respect to those matchings, until .
(Sufficiency) Assume is strongly connected and spanned by a disjoint union of cycles. It follows that also is strongly connected and condition (i) from Theorem 1 is satisfied. Since is spanned by a disjoint union of cycles, by Lemma 1, there exists a perfect matching (which is also a maximum matching) in . This implies condition (ii), i.e., .
In the proof of Lemma 2, the technical challenge is to show that the state vertices in the state-output bipartite graph cannot be always matched by an edge whose right-vertex is a sensor vertex (when the state graph is strongly connected), which implies that those states need to be always matched by edges whose end-points are state vertices. Therefore, by leveraging Lemma 1, it follows that the state graph has to be spanned by cycles.
V Main Results
Before we present the solution to the former problem, we need to review the notion of linking (see, for instance, ) from the vertices in the state digraph to the vertices in the communication digraph in the state-output graph associated with the augmented system. Specifically, a linking is a set of vertex-disjoint and simple paths in , from the vertices in to the vertices in . Additionally, for each sensor , we denote by the communication-linking (a linking where both the starting and ending vertices in the vertex-disjoint simple paths belong to the communication graph) from the set of sensor vertices that belong to the edges in a maximum matching of to a subset of in-neighbors of sensor () with equal cardinality. In particular, there are as many of those sensor vertices as left-unmatched vertices in a maximum matching associated to due to the observability of , and its structural observability, as prescribed by Theorem 1. Consequently, the solution to Problem 1 can be formally stated as follows.
(Necessity) It is enough to show that one condition from Theorem 2 or Theorem 3 does not hold when is not spanned by a disjoint union of cycles or that is not strongly connected. Therefore, assume that is structurally observable and is not strongly connected. The proof follows along the same lines as the proof of necessity in Lemma 2 since condition (i) of Theorem 2 fails if is not strongly connected. Now, assume that is structurally observable and is not spanned by a disjoint union of cycles. Using the second part of the proof of Lemma 2 for corresponding to each of the strongly connected components of proves the contradiction to condition (ii) of Theorem 2 if is not spanned by a disjoint union of cycles.
(Sufficiency) Assume is spanned by a disjoint union of cycles for all , and is strongly connected. In order to prove sufficiency, we follow similar steps to those in the proof in  and show that the same conditions are satisfied by a more general .
The necessity and sufficiency of condition (i) and sufficiency of condition (ii) in Theorem 2 follow as in , since is assumed to be strongly connected. The original system (1)-(2) is also structurally observable, and, from Theorem 1, we know there exists a maximum matching associated to such that, for every left-unmatched vertex , there is a distinct sensor associated to it. Expanding the maximum matching to the augmented system’s bipartite state graph , we can match all the vertices with a distinct sensor measuring it. This yields that the only possible left-unmatched vertices in are , hence, . All sensors that are not right-matched by a path from a previously unmatched state vertex are spanned by disjoint cycles, by the assumption on . Then, either these left-unmatched vertices are already in-neighbors of sensor , or, following a similar procedure as in the proof of Lemma 2, we can find another maximum matching such that .
Next, to show that there exists a realization of that ensures observability of , we leverage the proof of Theorem 3 in . Specifically, we invoke the Popov-Belevitch-Hautus criterion to assess the observability of the system . As a result, must be such that the following equalities hold for :
The structure of
does not allow arbitrary placing of the eigenvalues, as opposed to the proof of Theorem3 in . However, we are able to prove that the eigenvalues of that we cannot place do not affect the rank of . The eigenvalues associated with the cycles in can be arbitrarily placed: for each cycle, composed of edges with weights , where and is the number of cycles, the associated eigenvalues will have the values: . The eigenvalues that cannot be placed are associated to the paths and will be zero [18, 19]. The same analysis holds for , i.e., the zero eigenvalues are associated with the paths that are not spanned by cycles. More specifically, the vertices on these paths are exactly the left-unmatched vertices with respect to a maximum matching in . In the state-output digraph , in order to match these left-unmatched vertices, the paths are extended through the links described by to the sensors’ states, in the communication graph . Let be the number of vertices in the minimum length paths in and that are not spanned by cycles, i.e., corresponding to the zero eigenvalues in and . By suitable permutations, we can separate the blocks in and associated to the disjoint cycles, denoted symbolically by and the blocks associated to the linkings , respectively :
The eigenvalues in can be chosen to be different than the eigenvalues of , which are non-zero, and different than zero, hence has non-zero eigenvalues. Moreover, the end-vertices of the linkings are measured by the outputs given by . Therefore, the pair is observable and, by Popov’s criterion,
This completes the proof that rank, i.e., the conditions of Theorem 4 are sufficient.
Theorem 4 accounts for scenarios where the sensors are memoryless, i.e., they do not retain their previous state to integrate it in the overall dynamics. This extends the results in , revisited in Section III. Specifically, the case where sensors are not readily memoryless leads to the case where the communication graph is strongly connected and has a subgraph spanned by a disjoint union of cycles, since the access of a sensor to its state and incorporation in the overall dynamics corresponds to a self-loop in the communication graph, which is an elementary cycle.
In the context of limited communication decentralized estimation schemes that employ sensors which cannot discern between the contributions coming from their neighbors due to the technology used, i.e., when in (5), it follows that is at most rank . More specifically, there will be only one possible communication-linking ending at the ’th sensor vertex, implying that the state bipartite graph’s maximum matching can have at most one left-unmatched vertex.
Finally, given the necessary and sufficient conditions for the communication graph provided in Theorem 4, we aim to formulate the problem of designing a minimum cost communication graph, as stated in Problem 2, as an integer programming problem. To this end, we leverage problems such as the minimum cost maximum matching problem and minimum cost spanning trees . We also need to formulate the conditions on the communication graph, which require to encode the minimum cardinality linkings .
To better visualize the results, we write the communication graph as , where represents the set of sensor states, and the set of communication links between the sensors. Let be the communication cost incurred by establishing a link from sensor to sensor . If we want to obtain a communication graph dealing with memoryless sensors, then we prescribe , and obtain a finite cost graph as a feasible solution.
Briefly, the constraints that have to be satisfied by can be described in the following algorithm where steps are addressed simultaneously: for every sensor ,
1. Find the number of left-unmatched vertices in ;
2. Find the minimum cost linking ;
3. Run the minimum cost maximum matching algorithm on and select the edges that compose it; and
4. Add the minimum cost edges such that the graph is strongly connected.
We can leverage some insights provided by the heuristic algorithm provided in to obtain an integer programming problem formulation without explicitly computing . For a sensor , add a ‘virtual output’ (i.e., not part of the sensing technology but with the same role under this intermediate step) to each of its in-neighbors, according to . Denote this set of vertices by . Next, let be the state-output bipartite graph of the system with virtual outputs and the associated state-output digraph. We then expand the cost structure as follows: assign weights w.r.t. sensor to all the edges that are not in the communication graph and for . This setup ensures that the minimum cost maximum matching algorithm in the former state-output bipartite graph will return a matching that partitions the state-output digraph in vertex-disjoint paths and cycles, while incurring the minimum cost. Specifically, the paths will contain those described by , and the rest of the digraph will be spanned by disjoint cycles. Furthermore, notice that since there are no edges from the communication digraph to the state digraph, there can be no cycle spanning both vertices in and vertices in .
To state the integer programming formulation, let us denote by
the binary variable associated to the existence of edge. For a set , the cutset represents a subset of edges with the start vertex in and the end vertex in , for a given set of edges and set of vertices . Hence, the constraints are given by the matching problem on the state-output digraph and by the strong connectivity of the communication graph, which is imposed via rooted minimum spanning tree for each vertex. For brevity, denote . Thus, we obtain the following formulation of Problem 2:
where the edges in the minimum cost communication graph can be retrieved from .
The design problem proposed in Problem 2 is NP-hard, since it contains as a particular instance the design problem addressed in . Hence, a straightforward greedy algorithm can be implemented by sequentially performing the steps described in the pseudo-algorithm above. Nonetheless, the solution will depend on the initial point since, at each iteration, the previous selected edges in will be set to one, which does not guarantee that the final configuration of has indeed minimum cost. Consequently, we leverage the integer programming formulation which is also known to be NP-hard in general, but which we can solve by resorting to highly optimized off-the-shelf software toolboxes (e.g., YALMIP) that have been efficiently deployed in practice when dealing with large-scale complex problems [20, 21].
Vi Illustrative example
Consider the example in Figure 1 and associate the following cost matrix for the communication links:
The minimum topology for the communication graph such that decentralized observability from all sensors is ensured is depicted in Figure 2. Since there are two left-unmatched vertices in , each sensor should have at least two in-neighbors. Here, we illustrate how decentralized observability from sensor 1 is achieved. We need to find a linking such that is spanned by a disjoint union of cycles. In this case, pick a maximum matching in composed of the edges . The linking from the matched sensors to the in-neighbors of sensor 1 can be chosen as . We obtain that , which is trivially spanned by a union of disjoint cycles.
In this paper, we have extended the limited communication decentralized estimation schemes to cope with general scenarios and provided necessary and sufficient conditions for the communication graph such that retrieval of the state of the system and sensors is possible. In particular, the present extension enables the deployment of limited communication decentralized estimation schemes in the scenarios where the sensors are memoryless, and where the sensors do not have the capacity to discern the contributions and/or state of other sensors. Furthermore, we cast the design problem under communication costs, i.e., the problem of determining the minimum cost communication graph required to implement a limited communication decentralized estimation scheme, as an integer programming problem. This formulation enables the use of off-the-shelf software toolboxes that are reliable in practice when dealing with large-scale complex problems.
Future research will focus on proposing energy-efficient communication protocols between the sensors when subject to constrained energy budgets. Towards this goal, it is important to understand the trade-offs between communication and the information contained in the sensors states. In particular, we aim to quantify and classify the role of the sensors’ state dimension in the estimation process and accuracy, which can be key when adding communication infrastructure is prohibitive. Our findings suggest that as the number of dimensions of the exchanged states increases, fewer communication links are required to guarantee decentralized observability. Moreover, one can design the dimension of the sensors’s memory such that no additional links are necessary.
-  V. Gupta, “Distributed estimation and control in networked systems,” Ph.D. dissertation, California Institute of Technology, 2007.
-  Y.-F. Huang, S. Werner, J. Huang, N. Kashyap, and V. Gupta, “State estimation in electric power grids: Meeting new challenges presented by the requirements of the future grid,” IEEE Signal Processing Magazine, vol. 29, no. 5, pp. 33–43, 2012.
-  S. Kar, “Large scale networked dynamical systems: Distributed inference,” Ph.D. dissertation, Carnegie Mellon University, 2010.
-  U. A. Khan, “High-dimensional consensus in large-scale networks: Theory and applications,” Ph.D. dissertation, Carnegie Mellon University, 2009.
-  L. Wang and A. Morse, “A distributed observer for a time-invariant linear system,” in American Control Conference (ACC), 2017. IEEE, 2017, pp. 2020–2025.
-  A. Mitra and S. Sundaram, “Distributed observers for lti systems,” arXiv preprint arXiv:1608.01429, 2016.
-  F. Garin and L. Schenato, “A survey on distributed estimation and control applications using linear consensus algorithms,” in Networked Control Systems. Springer, 2010, pp. 75–107.
S. Das and J. M. Moura, “Distributed Kalman filtering with dynamic observations consensus,”IEEE Transactions on Signal Processing, vol. 63, no. 17, pp. 4458–4473, 2015.
-  M. V. Subbotin, “Distributed decentralized estimation,” Ph.D. dissertation, University of California at Santa Barbara, 2008.
-  U. A. Khan and A. Jadbabaie, “Coordinated networked estimation strategies using structured systems theory,” in 50th Conference on Decision and Control and European Control Conference. IEEE, 2011, pp. 2112–2117.
-  U. A. Khan, S. Kar, A. Jadbabaie, and J. M. Moura, “On connectivity, observability, and stability in distributed estimation,” in 49th IEEE Conference on Decision and Control. IEEE, 2010, pp. 6639–6644.
-  A. B. Alexandru, S. Pequito, A. Jadbabaie, and G. J. Pappas, “Decentralized observability with limited communication between sensors,” in 55th Conference on Decision and Control. IEEE, 2016, pp. 885–890.
-  J.-M. Dion, C. Commault, and J. Van Der Woude, “Generic properties and control of linear structured systems: a survey,” Automatica, vol. 39, no. 7, pp. 1125–1144, 2003.
-  C.-T. Lin, “Structural controllability,” IEEE Transactions on Automatic Control, vol. 19, no. 3, pp. 201–208, 1974.
-  R. Shields and J. Pearson, “Structural controllability of Multi-Input linear systems,” IEEE Transactions on Automatic control, vol. 21, no. 2, pp. 203–212, 1976.
-  S. Pequito, S. Kar, and A. P. Aguiar, “A framework for structural input/output and control configuration selection in large-scale systems,” IEEE Transactions on Automatic Control, vol. 61, no. 2, pp. 303–318, 2016.
-  R. Diestel, Graph theory, ser. Graduate Texts in Mathematics. Springer, 2005.
-  C. Coates, “Flow-graph solutions of linear algebraic equations,” IRE Transactions on circuit theory, vol. 6, no. 2, pp. 170–187, 1959.
-  K. J. Reinschke, Multivariable control: a graph theoretic approach. Springer-Verlag, 1988.
G. L. Nemhauser and L. A. Wolsey,
Integer programming and combinatorial optimization. Springer, 1988, vol. 20.
-  J. Lofberg, “YALMIP: A toolbox for modeling and optimization in MATLAB,” in International Symposium on Computer Aided Control Systems Design. IEEE, 2004, pp. 284–289.