Privacy-Preserving Average Consensus via State Decomposition

02/25/2019 ∙ by Yongqiang Wang, et al. ∙ Clemson University 0

Average consensus underpins key functionalities of distributed systems ranging from distributed information fusion, decision-making, distributed optimization, to load balancing and decentralized control. Existing distributed average consensus algorithms require each node to exchange and disclose state information to its neighbors, which is undesirable in cases where the state is private or contains sensitive information. In this paper, we propose a novel approach that avoids disclosing individual state information in average consensus by letting each node decompose its state into two sub-states. For each node, one of the two sub-states involves in computation and inter-node interactions as if it were the original state, while the other sub-state interacts only with the first sub-state of the same node, being completely invisible to other nodes. The initial values of the two sub-states are chosen randomly but with their mean fixed to the initial value of the original state, which is key to guarantee convergence to the desired consensus value. In direct contrast to differential-privacy based privacy-preserving average-consensus approaches which enable privacy by compromising accuracy in the consensus value, the proposed approach can guarantee convergence to the exact desired value without any error. Not only is the proposed approach able to prevent the disclosure of a node's initial state to honest-but-curious neighbors, it can also provide protection against inference by external eavesdroppers able to wiretap communication links. Numerical simulations demonstrate the effectiveness of the approach and its advantages over state-of-the-art counterparts.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

As a building block of distributed computing, average consensus has been an active research topic in computer science and optimization for decades [1, 2]. In recent years, with the advances of wireless communications and embedded systems, particularly the advent of wireless sensor networks and the Internet-of-Things, average consensus is finding increased applications in fields as diverse as automatic control, signal processing, social sciences, robotics, and optimization [3].

Conventional average consensus approaches rely on the exchange of explicit state values among neighboring nodes to reach agreement on distributed computation. Such a disclosure of state information has two potential problems. First, it breaches the privacy of participating nodes who may not want to disclose their state values containing sensitive and private information. For example, a group of individuals using average consensus to compute a common opinion may want to keep each individual’s opinion secret [4]. Another example is power systems where multiple generators want to reach agreement on cost while keeping their individual generation information private since the generation information is sensitive in bidding the right for energy selling [5]. Secondly, exchanging information through wireless or wired communications is vulnerable to eavesdroppers which try to steal information by tapping communication links. With the increased number of reported attack events, preserving data privacy has become an urgent need in many social and engineering applications.

To address the pressing need for privacy-preserving average consensus, one may resort to conventional secure multi-party computation approaches such as Yao’s Garbled Circuit [6], Shamir’s Secret Sharing algorithm [7], or many other recent advances [8]. However, such general-purpose privacy protecting approaches are both computationally and communicationally too heavy for systems with fast-evolving behaviors particularly cyber-physical systems which are subject to hard real-time constraints. For example, Yao’s Garbled Circuit has a computational latency on the order of seconds [9], whereas the tolerable computational latency is on the order of milliseconds for the real-time control of connected automated vehicles [10] and unmanned aerial vehicles [11]. Recently, several dedicated privacy-preserving solutions have been proposed for average consensus [12, 13, 14, 15, 16, 17, 18]. Most of these approaches rely on the idea of obfuscation to mask true state values by injecting carefully-designed noise on the states. One commonly used tool is differential privacy from the database literature in computer science [13, 14, 16, 17, 18]. However, obfuscation under differential privacy affects the accuracy of average consensus, preventing convergence to the exact desired value. Another tool emerged recently is the correlated-noise based obfuscation [15, 12], which can guarantee the accuracy of average consensus. Observability based approaches have also been discussed in the dynamics and control community to protect the privacy of multi-agent networks. The basic idea is to design the interaction topology to minimize the observability from a certain node, which amounts to minimizing the node’s ability to infer the initial states of other nodes in the network [19, 20, 21]. However, both the correlated-noise based and the observability based approaches are vulnerable to adversary nodes which are directly connected to a target node as well as all its neighbors [22].

To improve resilience to privacy attacks, another common approach is to employ cryptography. However, although cryptography based approaches can easily enable privacy preservation with the assistance of an aggregator or third-party [23], like in cloud-based control or computation [24, 25, 26], their extension to the completely decentralized average consensus problem in the absence of an aggregator or third-party is extremely hard due to the difficulties in decentralized key management. In fact, to our knowledge, except our recent result [27, 22], existing efforts ([28, 29]) incorporate cryptography into decentralized average consensus without giving participating nodes access to the final consensus value (note that in [29] individual participating nodes do not have access to the decryption key to decrypt the final consensus value which is obtained in the encrypted form, otherwise they will be able to decrypt intermediate computations to access other nodes’ states). Furthermore, cryptography based approaches will also significantly increase communication and computation overhead (please see e.g., [30] for detailed discussions), which is not appropriate for systems with limited resources or systems with fast evolving behaviors or subject to hard real-time constraints.

In this paper, we propose a state-decomposition based approach that can guarantee the privacy of all participating nodes in average consensus without compromising accuracy. Our basic idea is to let each node decompose its state into two sub-states with random initial values. One sub-state succeeds the role of the original state in inter-node interactions while the other sub-state only interacts with the first sub-state in the same node and thus is completely invisible to outside nodes. To ensure consensus to the right average value, the initial values of the two sub-states are randomly chosen but with their mean fixed to the initial value of the original state. Different from existing differential-privacy based approaches which sacrifice accuracy for privacy, our approach can guarantee convergence to the exact average consensus value. Unlike correlated-noise based or observability based approaches which require a node to have at least one neighbor that is not directly connected to the adversary to maintain privacy, our approach can guarantee privacy of a node even when the node and all its neighbors are directly connected to the adversary. Furthermore, the approach is completely decentralized and light-weight in computation, which makes it easily applicable to resource-restricted systems. Numerical simulation results are given to verify the results.

Ii Background

Ii-a Average Consensus

We first review the average consensus problem. Following the convention in [3], we represent a network of nodes as a graph with node set , edge set , and the adjacency matrix denoting coupling weights which satisfy if and 0 otherwise. Here is time index, denoting that could be time-varying. The set of neighbors of a node is denoted as and its cardinality is denoted as .

Throughout this paper we make the following assumption:

Assumption 1

We assume that the graph is undirected and connected, i.e., holds for all and there exists a (multi-hop) path between any pair of nodes.

We represent the state variable of a node as

. For the sake of simplicity, we assume scalar states. But as commented later in Remark 5, the results are easily extendable to the case where the state is a vector. To achieve average consensus, namely convergence of all states

to the average of initial values, i.e., , the update rule is formulated as [31]

(1)

where resides in the range with defined as

(2)

It has been well known that average consensus can be achieved if the network is connected and there exists some such that holds for all [32].

Ii-B Attack Model

In the paper, we consider two types of adversaries:

An honest-but-curious adversary is a node who follows all protocol steps correctly but is curious and collects received data in an attempt to learn some information about other participating nodes.

An eavesdropper is an external attacker who knows the network topology, and is able to wiretap communication links and access exchanged messages.

Generally speaking, an eavesdropper is more disruptive than an honest-but-curious node in terms of information breaches because it can snoop messages exchanged on many channels whereas the latter can only access the messages destined to it. However, an honest-but-curious node does have one piece of information that is unknown to an external eavesdropper, i.e., the internal initial state is available if node is an honest-but-curious node. We will systematically analyze the enabled privacy strength of our approach against both adversaries.

Iii Privacy-Preserving Approach

Fig. 1: State-decomposition based privacy-preserving average consensus. (a) Before state decomposition (b) After state decomposition

The key idea of our approach is a decomposition mechanism:

Decomposition Mechanism: We let each node decompose its state into two sub-states and , with the initial values and randomly chosen from the set of all real numbers under the constraint (cf. Fig. 1). The sub-state succeeds the role of the original state in inter-node interactions and it is in fact the only state value from node that can be seen by its neighbors. The other sub-state also involves in the distributed interaction by (and only by) interacting with . So the existence of is invisible to neighboring nodes of node , although it directly affects the evolution of . Taking node 1 in Fig. 1(b) for example, acts as if it were in the inter-node interactions while is invisible to nodes other than node , although it affects the evolution of . The coupling weight between the two sub-states and is symmetric and denoted as . It is a design parameter and will be elaborated later in the Weight Mechanism.

Under the state-decomposition approach, the overall dynamics become

(3)

subject to .

Remark 1

Compared with (1), since every “visible” sub-state’s number of neighbors is increased by 1, the upper bound on is reduced from to with defined in (2).

In conventional average consensus algorithms, the coupling weights are required to be within , which restricts the strength of achievable privacy (as will be clear from the proof of Theorem 2). We introduce the following weight mechanism to enable strong privacy:

Weight Mechanism: For , we allow all weights and to be arbitrarily chosen from the set of all real numbers under the constraint ; For , we require that there exists a scalar such that all nonzero satisfy and all satisfy .

In the following, we first prove that under the approach, i.e., the Decomposition Mechanism and the Weight Mechanism, all states and will converge to the same average consensus value as in the conventional case (1). Then we rigorously analyze the privacy of participating nodes enabled by the proposed approach in the presence of an eavesdropper or honest-but-curious node.

Theorem 1

Under Assumption 1 and the Weight Mechanism, all sub-states in (3) converge to the average consensus value of (1), i.e.,

(4)

Proof: Under the symmetric weight assumption and , one can easily obtain that for the network after decomposition, the sum of all sub-states are always time-invariant. Therefore, even the weights are allowed to be arbitrarily chosen from the set of all real numbers for , we always have

(5)

Starting from , as all coupling weights compose a connected graph, the Decomposition Mechanism and the Weight Mechanism guarantee that all sub-states also compose a connected graph. According to the result on average consensus under time-varying weights [32], average consensus can still be achieved, i.e., all sub-states and will converge to , which is equal to according to (5). Further making use of the fact leads to the conclusion that all sub-states converge to .

Remark 2

For the purpose of privacy-preservation, the values of should be private to node .

Next we rigorously analyze the enabled privacy against an honest-but-curious adversary or an external eavesdropping adversary. To this end, we first give a definition of privacy.

Definition 1

The privacy of the initial value of node

is preserved if an adversary cannot estimate the value of

with any guaranteed accuracy.

Definition 1 requires that an adversary cannot even find a range for a private value and thus is more stringent than the privacy preservation definition considered in [12, 15] which defines privacy preservation as the inability of an adversary to uniquely determine the protected value. Next we show that even by carefully observing a node’s communication for multiple steps, an adversary cannot infer the node’s initial state with any guaranteed accuracy.

Theorem 2

Under the Decomposition Mechanism and the Weight Mechanism, an honest-but-curious node cannot infer the initial state of node with any guaranteed accuracy if node has at least one neighboring node who does not collude with node to infer (cf. Fig. 2 for an illustrative example).

Proof:

Fig. 2: Connection configuration example for Theorem 2.

To prove that node cannot estimate with any guaranteed accuracy, we show that any arbitrary variation of is indistinguishable to node , i.e., the information accessible to node can be exactly the same even if were changed to an arbitrary value . We define the information accessible to the honest-but-curious node at iteration as . So as time evolves, the cumulated information accessible to node can be summarized as .

To show that the privacy of the initial value can be preserved against node , i.e., node cannot estimate the value of with any guaranteed accuracy, it suffices to show that under any initial value the information accessible to node , i.e., could be exactly the same as , the cumulated information accessible to node under . This is because the only information available for node to infer the initial value is , and if could be the outcome under any initial values of , then node has no way to even find a range for the initial value . Therefore, we only need to prove that for any , could hold.

Next we show that there exist initial values of and coupling weights satisfying the requirements of the Weight Mechanism that make hold under . (Note that the alternative initial values of should guarantee that the agents still converge to the original average value after is altered to .) More specifically, under the following initial condition

(6)

and coupling weights

(7)

where “” represents set subtraction, it can be easily verified that holds for any . Note that the first equation in (6) is used to guarantee that the consensus value does not change under the alternative initial values and . Therefore, the honest-but-curious node cannot learn the initial state of node based on accessible information if node is also connected to another node that does not collude with node to infer (note that node is allowed to exchange information with the honest-but-curious node following the protocol, as illustrated in Fig. 2).

Remark 3

In the derivation, the choice of coupling weights , , and in (7) guarantees that starting from , all sub-states under the alternative initial value will be the same as those under the original initial value , and hence all coupling weights can be the same starting from under the two different initial value conditions. Depending on the value of , the weights , , and could be outside the range , which corroborates the necessity of allowing weights at to be arbitrarily chosen from the set of all real numbers in the Weight Mechanism. Note that since the sub-states and are also arbitrarily chosen from the set of all real numbers, the possibility of them making the denominators in (7) equal to zero is negligible.

Remark 4

Our approach can protect the privacy of node even when node and all its neighbors are directly connected to the honest-but-curious neighbor (cf. Fig. 2), which is not allowed in the privacy-preserving approaches in [12, 15]. This illustrates the advantage of the proposed state-decomposition based approach.

Similar results can be obtained for the eavesdropping adversary case:

Theorem 3

Under the Decomposition Mechanism and the Weight Mechanism, an eavesdropper cannot infer the initial state of any node with any guaranteed accuracy if node has at least one neighboring node whose interaction weight with node is inaccessible to the eavesdropper.

Proof: Following the line of reasoning in Theorem 2, we can obtain that any change in the initial value can be completely compensated by changes in , , and that are invisible to the eavesdropper. Therefore, the accessible information to the eavesdropper can be exactly the same even when were changed arbitrarily and hence the eavesdropper cannot infer the initial value of node based on accessible information.

Remark 5

The results are also applicable in the vector-state case. In fact, as long as the scalar state elements in the vector state have independent coupling weights, privacy can be naturally enabled in the vector-state case by applying results in this paper to individual scalar state elements.

Iv Numerical Comparison with Existing Results

In this section, we numerically compare our state-decomposition based privacy-preserving approach with existing state-of-the-art counterparts [12, 15, 18] to confirm its advantages.

For the convenience in comparison, we represent the internal state of node as and its obfuscated version (used in state exchange with neighbors) in existing obfuscation based approaches as . We considered a network of five nodes with interaction topology and weights given in [12], which, under our formulation framework translate into and

(8)

Without loss of generality, we suppose that an external eavesdropper is interested in obtaining the initial state of node 1 and constructs the following observer to infer :

(9)

where the initial observer state is set to . We assume that the eavesdropper has access to all weights except which was set to a random value 0.7 in the step of the observer.

Fig. 3 gives the evolution of the network states as well as the eavesdropper’s observer state under the privacy-preserving approach in [12]. The initial states of five nodes were set to . It can be seen that although convergence to the right average value is achieved, the initial internal state of node 1, i.e., , can also be inferred by the eavesdropper’s observer .

Similar results were obtained using the approach in [15], which guaranteed accurate average consensus but not the privacy of (cf. Fig. 4).

Fig. 3: An eavesdropper can infer the initial state of node 1 under the privacy protocol in [12].

Based on the same setup, we also simulated the proposed state-decomposition based privacy-preserving approach. The coupling weights at were randomly chosen from . The results are given in Fig. 5, which confirms that the proposed approach can protect the privacy of all nodes’ initial values against an eavesdropper while achieving accurate average consensus.

It is worth noting that although differential-privacy based approaches such as [18] can also protect the privacy of participating nodes’ initial values, they also lead to errors in the final consensus value, as confirmed by the simulation results in Fig. 6. In such approaches, due to the trade-off between privacy and accuracy, when an application calls for higher accuracy of the consensus result, the risk of disclosing one’s initial state also becomes higher.

Fig. 4: An eavesdropper can infer the initial state of node 1 under the privacy protocol in [15].
Fig. 5: The proposed approach can achieve accurate average consensus while avoiding an eavesdropper from inferring the initial state of node 1.
Fig. 6: Although by using a high level of noise, the differential privacy approach in [18] can avoid an eavesdropper from accurately estimating the initial state of node 1 (with large estimation error ), its achievable accuracy in average consensus also deteriorates with an increased noise level, as reflected by a larger consensus error .

V Conclusions

In this paper, we proposed a privacy-preserving approach for the network average consensus problem based on state decomposition. In contrast to differential-privacy based approaches which are subject to a fundamental trade-off between enabled privacy and achievable consensus accuracy, the approach is able to enable privacy preservation while guaranteeing accurate average consensus. It is also superior to correlated-noise based obfuscation approaches which can guarantee accurate average consensus but not resilience to adversaries which are directly connected to a target node as well as all its neighbors. Simulation results confirmed the theoretical predictions.

References

  • [1] M. H. Degroot. Reaching a consensus. Journal of the American Statistical Association, 69(345):118–121, 1974.
  • [2] N. A. Lynch. Distributed Algorithms. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1996.
  • [3] R. Olfati-Saber, J. A. Fax, and R. M. Murray. Consensus and cooperation in networked multi-agent systems. Proceedings of the IEEE, 95(1):215–233, 2007.
  • [4] J. N. Tsitsiklis. Problems in decentralized decision making and computation. PhD thesis, 1984.
  • [5] X. Fang, S. Misra, G. Xue, and D. Yang. Smart grid – the new and improved power grid: A survey. IEEE communications surveys & tutorials, 14(4):944–980, 2012.
  • [6] A. C. Yao. Protocols for secure computations. In Proceedings of the 23rd Annual Symposium on Foundations of Computer Science, pages 160–164, 1982.
  • [7] A. Shamir. How to share a secret. Communications of the ACM, pages 612–613, 1979.
  • [8] M. Prabhakaran and A. Sahai (eds). Secure Multi-party Computation. IOS press, 2013.
  • [9] B. Kreuter, A. Shelat, and C. Shen. Billion-gate secure computation with malicious adversaries. In USENIX Security Symposium, volume 12, pages 285–300, 2012.
  • [10] L. P. Clare, J. L. Gao, E. H. Jennings, and C. Okino. A network architecture for precision formation flying using the IEEE 802.11 MAC protocol. In Proceedings of 2005 IEEE Aerospace conference, pages 1335–1347, 2005.
  • [11] W. Chen and S. Cai. Ad hoc peer-to-peer network architecture for vehicle safety communications. IEEE Communications Magazine, 43(4):100–107, 2005.
  • [12] Y. Mo and R. M. Murray. Privacy preserving average consensus. IEEE Transactions on Automatic Control, 62(2):753–765, Feb 2017.
  • [13] E. Nozari, P. Tallapragada, and J. Cortés. Differentially private average consensus with optimal noise selection. IFAC-PapersOnLine, 48(22):203 – 208, 2015.
  • [14] Z. Huang, S. Mitra, and N. Vaidya. Differentially private distributed optimization. In Proceedings of the 2015 International Conference on Distributed Computing and Networking, ICDCN ’15, New York, NY, USA, 2015. ACM.
  • [15] N. E. Manitara and C. N. Hadjicostis. Privacy-preserving asymptotic average consensus. In 2013 European Control Conference (ECC), pages 760–765, July 2013.
  • [16] E. Nozari, P. Tallapragada, and J. Cortés. Differentially private average consensus: obstructions, trade-offs, and optimal algorithm design. Automatica, 81(7):221–231, 2017.
  • [17] V. Katewa, F. Pasqualetti, and V. Gupta. On privacy vs cooperation in multi-agent systems. International Journal of Control, (accepted):1–20, 2017.
  • [18] Z. Huang, S. Mitra, and G. Dullerud. Differentially private iterative synchronous consensus. In Proceedings of the 2012 ACM workshop on Privacy in the electronic society, pages 81–90. ACM, 2012.
  • [19] S. S Kia, J. Cortés, and S. Martínez. Dynamic average consensus under limited control authority and privacy requirements. International Journal of Robust and Nonlinear Control, 25(13):1941–1966, 2015.
  • [20] S. Pequito, S. Kar, S. Sundaram, and A. P. Aguiar. Design of communication networks for distributed computation with privacy guarantees. In In Decision and Control (CDC), 2014 IEEE 53rd Annual Conference on, pages 1370–1376, 2014.
  • [21] A. Alaeddini, K. Morgansen, and M. Mesbah. Adaptive communication networks with privacy guarantees. In Proceedings of 2017 American Control Conference, pages 4460–4465, 2017.
  • [22] M. Ruan, H. Gao, and Y. Q. Wang. Secure and privacy-preserving consensus. IEEE Transactions on Automatic Control, in press, 2019.
  • [23] N. Gupta and N. Chopra. Confidentiality in distributed average information consensus. In 2016 IEEE 55th Conference on Decision and Control (CDC), pages 6709–6714, Dec 2016.
  • [24] K. Kogiso and T. Fujita. Cyber-security enhancement of networked control systems using homomorphic encryption. In Proceedings of the 54th IEEE International Conference on Decision and Control, pages 6836–6843, 2015.
  • [25] Y. Shoukry, G. Konstantinos, A. Alanwar, G. J. Pappas, S. A. Seshia, M. Srivastava, and P. Tabuada. Privacy-aware quadratic optimization using partially homomorphic encryption. In Proceedings of the 55th IEEE International Conference on Decision and Control, pages 5053–5058, 2016.
  • [26] R. L. Lagendijk, Z. Erkin, and M. Barni. Encrypted signal processing for privacy protection: Conveying the utility of homomorphic encryption and multiparty computation. IEEE Signal Processing Magazine, 30(1):82–105, Jan 2013.
  • [27] M. Ruan, M. Ahmad, and Y. Q. Wang. Secure and privacy-preserving average consensus. In Proceedings of the 2017 Workshop on Cyber-Physical Systems Security and PrivaCy, pages 123–129. ACM, 2017.
  • [28] Moreno Ambrosin, Paolo Braca, Mauro Conti, and Riccardo Lazzeretti. ODIN: Obfuscation-based privacy-preserving consensus algorithm for decentralized information fusion in smart device networks. ACM Transactions on Internet Technology, 18(1):6, 2017.
  • [29] N. M. Freris and P. Patrinos. Distributed computing over encrypted data. In 2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pages 1116–1122, Sept 2016.
  • [30] C. Zhang, M. Ahmad, and Y. Q. Wang. ADMM based privacy-preserving decentralized optimization. IEEE Transactions on Information Forensics and Security, 14(3):565–580, 2019.
  • [31] Reza Olfati-Saber, J Alex Fax, and Richard M Murray. Consensus and cooperation in networked multi-agent systems. Proceedings of the IEEE, 95(1):215–233, 2007.
  • [32] A. Nedic, A. Ozdaglar, and P. A. Parrilo. Constrained consensus and optimization in multi-agent networks. IEEE Transactions on Automatic Control, 55(4):922–938, 2010.