Information and signal fusion in multi-sensor systems has recently gained a significant attention in signal-processing literature [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]. Emergence of Internet of Things (IoT) and the so called Cyber-Physical Systems (CPS) has motivated practical applications, where sensors, and generally smart devices are integrated into the physical systems . Recently a growing interest is directed towards distributed estimation in CPS [6, 7, 3, 4, 5, 14, 15, 16, 2, 1], where the state of the dynamical system is globally tracked by sensors. The sensors communicate and share their estimations/measurements over the communication network at the same time-scale as the time-evolution of system dynamics. This scenario is known as single time-scale estimation, and is privileged with low communication demand and no constraint on local observability of sensors (see [6, 7, 3, 4, 5] for details), in contrast to kalman consensus filters [17, 9]. In this scenario, the system structure dictates the sensor-network structure. If the system is structurally full rank, a strongly-connected sensor network is sufficient for stable estimation [4, 5, 2, 1, 15]. However, more than strong connectivity is required for tracking rank-deficient systems, where it is shown that centralized communication hubs are required to recover observability by the monitoring cyber-network [4, 5, 14]. Indeed, (distributed) observability of the system determines the underlying communication topology among sensors/estimators111In this paper sensor and estimator are used interchangeably, referring to distributed local units measuring and tracking the state of dynamic system. in CPS.
Contributions: The questions that we address here are as follows: how the failure of a sensor affects the distributed observability in the single time-scale distributed estimation? And, what are the counter-measures to recover for this failure? These questions are to a great extent unexplored in the literature. It is known that if the sensor is essential for estimation its failure implies loss of necessary information and loss of system observability. We propose to overcome this problem by placing new sensors to measure new states in the dynamical system. The idea is to infer equivalent
information of the system to regain distributed observability and to stabilize the Mean Squared Error (MSE) of estimation. The sensors are classified into-type and -type based on their role in distributed estimation, see e.g., [4, 5] for details. Using a graph-theoretic approach, we show that if the failed sensor is -type measuring state , the observability is recovered by adding a new sensor measuring a state sharing a contraction with in system digraph. A polynomial order algorithm for contraction detection is provided. On the other hand, if the failed sensor is -type measuring state , the estimation is recovered by adding a new sensor measuring state sharing a Strongly Connected Component (SCC) with in system digraph. Note that, the proposed methods are graph-theoretic and irrespective of the numerical system parameters, that are particularly significant for large-scale system application.
Ii Problem Formulation
Consider estimation of an LTI system with measurements:
where is state-space,
as measurement vector,and , as noise variables with standard assumptions on Gaussianity and independence, and is the discrete-time index. Based on Kalman , estimation of the noise-corrupted LTI model in Eqs. (1)-(2) renders bounded estimation error if and only if the system is observable (detectable)222In this paper, it is assumed that there are no stable unobservable modes in the system. Therefore, detectability and observability are equivalent.. Observability ensures that the entire system state, , can be uniquely recovered from the noise-corrupted measurements, .
In this paper, single time-scale distributed estimation is addressed. In single time-scale distributed estimation, both system dynamics and distributed estimator evolve at the same time-scale [6, 7, 2, 1, 3, 4, 5, 14, 15]. This method is privileged with tracking faster system dynamics and imposing less communication burden. This is contrary to the two time-scale method where information-fusion (consensus) is performed faster than system dynamics [17, 9]. In the single-time estimation method, two types of information fusion are performed: (i) prediction-fusion, i.e. sharing sensors’ prediction of the state over a communication network; and (ii) measurement fusion, i.e. sharing sensors’ measurements over a communication network. Either one of these two types of information-fusion (for example only prediction fusion  or only measurement fusion 333Note that the case of only measurement fusion needs a fully-connected sensor network, and therefore, is a trivial case (not considered in this paper).) or both of them [4, 5] can be applied. Accordingly, the sensors are classified based on performing each type of information-fusion as follows:
Sensors are type- if they are required to share their measurement over communication network. Sensors are type- if they are required to share their state-prediction over communication network.
In this direction, the communication network of sensors are defined as follows:
Sensors perform prediction-fusion over network , with defined as the neighborhood of nodes and matrix as its adjacency matrix444Note that the weights in the adjacency matrix are defined such that is row-stochastic, see [4, 5, 15] for details.. Sensors perform measurement fusion over network , with defined as the neighborhood of nodes.
Further define as:
Given a system matrix , measurement matrices , communication network and , if is observable555We refer -observability as distributed observability. then there exist a gain matrix such that the distributed estimation achieves asymptotic omniscience and the error dynamics achieves global asymptotic stability on Mean Squared Error (MSE).
The problem we address in this paper is when one (or more) sensor fail to measure a state, and therefore, the system is not observable to the sensor network, i.e. the distributed estimator loses distributed observability. The countermeasure is to add a new sensor in the sensor network to recover for loss of observability, while the method for observability recovery and necessary connectivity in the sensor network differs depending on the type of sensor ( or ). Overall, the distributed observability recovery in sensor networks is defined in the following.
Problem Statement: Given system matrix , measurement matrices , communication network and , if sensor taking measurement fails then how to recover -observability with a new sensor measurement .666Note that here without loss of generality one sensor failure is considered. The results can be easily extended to the case of more than one sensor failure.
Iii Graph-theoretic Observability
To solve the problem, instead of the algebraic Grammian observability test, an alternative graph-theoretic approach is applied over system digraph. We first introduce some graph notions to present the main result on graph-theoretic observability and classification. Let and denote the set of state and measurement nodes, respectively. The system digraph is a directed adjacency graph defined as , with as the set of edges/links; a link, , in exists from to if . Similarly, a link, , in exists from to if . A path from to (or ) is denoted as . A path is called -connected (denoted by ) if it ends in . A cycle is a path starting and ending at the same node. A cycle-family is a set of cycles which are mutually disjoint and do not share any node. Similarly, a path and a cycle are disjoint if they do not share any node. More details on this construction can be found in [4, 5, 20].
A system is structurally-observable if and only if in its digraph :
Every state is included in a -connected path, i.e. ;
Every state is included in a disjoint family of -connected paths and cycles spanning .
Next we review some graph notions on the bipartite representation of system to find the necessary states for observability. A bipartite graph, , consists of two disjoint sets of nodes and with set of links starting in and ending in . The set of links and nodes are defined as follows: , , and the link set, defined as the collection of , if . Define a matching, denoted by , in the bipartite graph, , as the subset of links chosen such that they do not share an end-node in or a start-node in . This implies that the links in are mutually disjoint. The size (cardinality) of the matching, denoted by , is the number of links in . A maximum matching, , is a matching with maximum size. Note that maximum matching is not unique, in general. Given a matching, , denote by the set of matched nodes incident to in . The set of unmatched nodes, denoted by , is the set of nodes in not belonging to , i.e., . Next, define a Strongly Connected Component (SCC) as a subgraph in which every node is reachable via a directed path to every other node, i.e. for every we have . A parent SCC is defined as a SCC with no outgoing links to any nodes not belonging to that SCC, i.e. for every there is no such that . Having these graph-theoretic notions we are ready to present the following theorem:
The system (1) is observable if and only if the set of following state nodes in system digraph are measured:
(i) every unmatched state node ,
(ii) at least one state in every parent SCC in .
Following Definition 1 the sensors are classified according to the following lemma:
A sensor measuring an unmatched state node in system digraph is -type. A sensor measuring a state in a parent SCC is -type.
Iv Sensor Failure Recovery
Sensor failures are prevalent in sensor networks due to harsh environmental conditions, possible adversary attacks, and the nature of sensing devices . In this section, we investigate the countermeasures to recover distributed observability (-observability) under sensor failure. We separately analyze sensor failure in for measurement update and in for prediction fusion and the countermeasures to recover for distributed observability by adding equivalent sensor measurements.
Iv-a Type- Sensor Failure
Recall from Lemma 1 that an -sensor measures an unmatched state in system digraph and share its measurement (as a hub) over . Having a failed -sensor implies the loss of (distributed) observability of an unmatched state. Let represents the failed measurement, therefore, according to definition, must be recovered. The solution is by adding a new state measurement of structurally equivalent unmatched state. The equivalent unmatched state nodes are defined by the set of contractions in system digraph.
Contraction sets: Denote by the auxiliary graph associated to a matching, , and the bipartite graph, . Given , this graph is made by reversing all the links in while preserving the direction of links in . Given the matching and auxiliary graph , an -alternating path, , is constructed by sequence of links alternating between unmatched links, , and matched links, . The sequence starts with an unmatched link in from a free node in and every second link in . Given the matching and auxiliary graph , an -augmenting path, , is an alternating path with start node and end node in . Then, the contraction in the system is defined as follows:
Given a maximum matching, , for every unmatched node, , in auxiliary graph, , find the set of all nodes in reachable by alternating paths, , from . This is called the set of contraction nodes, denoted by .
Notice that in Algorithm 1, represents the XOR operator in set theory. As a result of this operator, each augmenting path increases the size of the matching by one till it reaches the maximum matching. More detailed explanations and definitions on contractions and related topics is available in .
Note that the connectivity of the new sensor in is similar to the connectivity of the failed sensor .
Iv-B Type- Sensor Failure
Recall from Lemma 1 that a -sensor measures a state in a parent SCC and shares its state prediction on a strongly-connected graph . Having a failed -sensor implies the loss of necessary information on state prediction. Let represents the failed sensor measurement, therefore according to Definition 2, the must be recovered. This is by an equivalent state measurement providing equivalent state prediction. These equivalent states are defined by the set of states sharing a SCC as discussed in Section III. Theorem 3 part-(ii) states that measurement of at least one state (any state) from each parent SCC is necessary for observability. Therefore, losing the measurement of one state in a parent SCC, the observability is recovered by measuring another state in the same SCC. This is done via well-known graph-theoretic algorithms for SCC classification, namely Depth First Search (DFS)  or Tarjan algorithm . These algorithms are widely provided in related literature [27, 28, 25] and are omitted here due to space limitation.
The size of parent SCC determines the number of equivalent states. If the parent SCC is in the form of a self-cycle, it only includes one state node and therefore there is no equivalent state measurement to recover for estimation.
Note that the new sensor follows similar connectivity as the failed sensor , i.e. it shares its prediction over a strongly-connected graph .
V Illustrative Example and Simulation
We consider a discrete-time dynamic system of states with random entries in , such that the given dynamic system is unstable with spectral radius . This ensures that the stability of tracking error is not trivial. Following the results of Theorem 3, we find the SCCs and unmatched node of the system. From Theorem 3 number of necessary sensors for observability is , one measuring the unmatched state and the other two each measuring a state in a parent SCC (see Fig.1-Left). Zero-mean Gaussian noise with covariance for both measurement and system is considered. Following distributed estimator is applied:
with defined as estimation of by sensor at step given the measurements up to time . The first step (3) is prediction fusion and the latter step (4) is measurement fusion. For this estimator the error dynamics is given as the following equation :
where is global error, and collects the noise terms. Note that, to have a fully distributed estimator, the gain matrix is block diagonal where the th diagonal block is the gain matrix for the estimator .
For distributed estimation and stability of the MSE according to Remark 1, -sensor directly shares its measurement of system unmatched state and two -sensors share their state predictions over a strongly-connected network. This network setting guarantees that the pair is observable, for which a block-diagonal gain matrix may be found using the procedure described in [29, 30], also see our prior work in . The weights of the prediction fusion network are chosen randomly, but such that
remains a stochastic matrix. The simulation is performed overMonte-Carlo trials and the time-evolution of the Mean Squared Error (MSE) at each sensor is shown in Fig.1-Right.
Next, we assume that the -sensor fails; therefore is not observable anymore and the tracking MSE goes unbounded for all sensors. As a countermeasure to recover for this, another sensor measuring an equivalent unmatched state node needs to be added. In Fig.2-TopLeft, using Algorithm 1 the set of equivalent unmatched nodes (contraction nodes) are shown in red color. Placing a sensor on the equivalent state and sharing its measurement directly with two other sensors recovers the distributed observability. This implies the stability of MSE for all sensors as shown in Fig.2-TopRight. Note that the design procedure for gain matrix is the same, as in [4, 29, 30]. Next, consider both -sensors are failed. Therefore, sensor measurements from equivalent states (sharing SCC) are required. This is given in Fig.2-BottomLeft, as states sharing a parent SCC are shown in different colors using DFS algorithm. Measuring an equivalent state in each SCC and sharing state predictions over a strongly-connected network guarantee the stability of MSE as shown in Fig.2-BottomRight.
Vi Concluding Remarks
Distributed estimation recovery via equivalent sensor measurements is addressed. Graph-theoretic algorithms are proposed to find the equivalent states for recovery of two types of sensors. The complexity of the Contraction Detection algorithm for -sensor recovery is . For SCC classification and -sensor recovery the complexity is . The polynomial order solution and the graph theoretic approach supports application in large-scale systems.
-  F. S. Cattivelli and A. H. Sayed, “Distributed detection over adaptive networks using diffusion adaptation,” IEEE Transactions on Signal Processing, vol. 59, no. 5, pp. 1917–1932, May 2011.
-  S. Tu and A. Sayed, “Diffusion strategies outperform consensus strategies for distributed estimation over adaptive networks,” IEEE Transactions on Signal Processing,, vol. 60, no. 12, pp. 6217–6234, 2012.
-  S. Kar and J. M. F. Moura, “Consensus+ innovations distributed inference over networks: cooperation and sensing in networked systems,” IEEE Signal Processing Magazine, vol. 30, no. 3, pp. 99–109, 2013.
-  M. Doostmohammadian and U. Khan, “On the genericity properties in distributed estimation: Topology design and sensor placement,” IEEE Journal of Selected Topics in Signal Processing, vol. 7, no. 2, pp. 195–204, 2013.
-  M. Doostmohammadian and U. Khan, “Graph-theoretic distributed inference in social networks,” IEEE Journal of Selected Topics in Signal Processing, vol. 8, no. 4, pp. 613–623, Aug. 2014.
S. Das and J. M. F. Moura,
“Consensus+ innovations distributed kalman filter with optimized gains,”IEEE Transactions on Signal Processing, vol. 65, no. 2, pp. 467–481, 2017.
-  S. Das and J. M. F. Moura, “Distributed kalman filtering with dynamic observations consensus,” IEEE Transactions on Signal Processing, vol. 63, no. 17, pp. 4458–4473, 2015.
-  M. Doostmohammadian and U. A. Khan, “Vulnerability of CPS inference to DoS attacks,” in 48th Annual Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, Nov. 2014, pp. 2015–2018.
-  E. J. Msechu, S. D. Roumeliotis, A. Ribeiro, and G. B. Giannakis, “Decentralized quantized Kalman filtering with scalable communication cost,” IEEE Transactions on Signal Processing, vol. 56, no. 8, pp. 3727–3741, Aug. 2008.
-  I. D. Schizas, A. Ribeiro, and G. B. Giannakis, “Consensus in ad hoc WSNs with noisy links part I: Distributed estimation of deterministic signals,” IEEE Transactions on Signal Processing, vol. 56, no. 1, pp. 350–364, 2008.
-  U. A. Khan and M. Doostmohammadian, “A sensor placement and network design paradigm for future smart grids,” in 4th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, San Juan, Puerto Rico, Dec. 2011, pp. 137–140.
-  M. Doostmohammadian and U. A. Khan, “On the distributed estimation of rank-deficient dynamical systems: A generic approach,” in 38th IEEE International Conference on Acoustics, Speech, and Signal Processing, Vancouver, CA, May 2013, pp. 4618–4622.
-  “Trillion sensors summit 2013, stanford university,” http://www.tsensorssummit.org/.
-  T. Boukhobza, F. Hamelin, S. Martinez-Martinez, and D. Sauter, “Structural analysis of the partial state and input observability for structured linear systems: Application to distributed systems,” European Journal of Control, vol. 15, no. 5, pp. 503–516, Oct. 2009.
-  S. Park and N. Martins, “Necessary and sufficient conditions for the stabilizability of a class of LTI distributed observers,” in 51st IEEE Conference on Decision and Control, 2012, pp. 7431–7436.
-  G. Battistelli, L. Chisci, G. Mugnai, A. Farina, and A. Graziano, “Consensus-based algorithms for distributed filtering,” in 51st IEEE Conference on Decision and Control, 2012, pp. 794–799.
-  R. Olfati-Saber, “Distributed Kalman filters with embedded consensus filters,” in 44th IEEE Conference on Decision and Control, Seville, Spain, Dec. 2005, pp. 8179–8184.
-  R. Kalman and R. Bucy, “New results in linear filtering and prediction theory,” ASME Journal of Basic Engineering, vol. 83, pp. 95–108, 1961.
-  M. Doostmohammadian and U. A. Khan, “On the characterization of distributed observability from first principles,” in 2nd IEEE Global Conference on Signal and Information Processing, 2014, pp. 914–917.
-  C. Godsil and G. Royle, Algebraic graph theory, New York: Springer, 2001.
-  Y. Y. Liu, J. J. Slotine, and A. L. Barabási, “Observability of complex systems,” Proceedings of the National Academy of Sciences, vol. 110, no. 7, pp. 2460–2465, 2013.
-  C. Lin, “Structural controllability,” IEEE Transactions on Automatic Control, vol. 19, no. 3, pp. 201–208, Jun. 1974.
-  Y. Y. Liu, J. J. Slotine, and A. L. Barabási, “Controllability of complex networks,” Nature, vol. 473, no. 7346, pp. 167–173, May 2011.
-  J. Yick, B. Mukherjee, and D. Ghosal, “Wireless sensor network survey,” Computer networks, vol. 52, no. 12, pp. 2292–2330, 2008.
-  K. Murota, Matrices and matroids for systems analysis, Springer, 2000.
-  J. Hopcroft and R. Karp, “An algorithm for maximum matchings in bipartite graphs,” SIAM Journal of Computation, pp. 225–231, 1973.
-  T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to Algorithms, MIT Press, 2009.
-  R. Tarjan, “Depth-first search and linear graph algorithms,” SIAM Journal on Computing, vol. 1, no. 2, pp. 146–160, 1972.
-  L. El Ghaoui, F. Oustry, and M. Ait Rami, “A cone complementarity linearization algorithm for static output-feedback and related problems,” IEEE Transactions on Automatic Control, vol. 42, no. 8, pp. 1171–1176, 1997.
-  O. L. Mangasarian and J. S. Pang, “The extended linear complementarity problem,” SIAM Journal on Matrix Analysis and Applications, vol. 2, pp. 359–368, Jan. 1995.