The recent push towards increasing connectivity of devices is enabling control applications to span wider geographical areas such as cities or even countries as in the case of smart grids. This increase in the number of available sensors, actuators and data has led to an increase in the controllers’ required computational capacity for such applications. One solution that has gained important traction is to outsource the computations to powerful remote servers, generically called clouds. Cloud-based computation services offer the potential to aggregate and to process information that is gathered from a large number of distributed agents, in domains such as machine learning, signal processing, smart grids, autonomous vehicles and control over the Internet-of-Things. Notwithstanding, there are substantial challenges that arise from the privacy-sensitive nature of some of the collected data. Secure communication to and from the cloud is a crucial aspect, but is not sufficient to protect the users’ data. Recent examples such as data leakage and abuse by aggregating servers[1, 2, 3] have drawn attention to the risks of storing data in the clear and urged measures against an untrustworthy cloud. A plethora of different schemes addressing this challenge have been proposed in the literature, spanning many research areas.
Schemes that allow secure function evaluation on private inputs from multiple parties while preserving the privacy of the data involved have been a desirable technique for many decades, coining the term secure multiparty computation. Some secure multiparty computation protocols provide computational security, while other provide information theoretic security. Computational security guarantees that there is no computationally efficient algorithm the parties involved in the protocol can implement so that they gain private information beyond what can be inferred from their inputs and outputs of the protocol only. On the other hand, information theoretic security guarantees that no algorithms of unlimited computational power exist that can lead to private data leakage. One approach within this context is to encrypt the computation itself. This primarily refers to Yao’s garbled circuits, introduced in , and the protocol of Goldreich-Micali-Widgerson, introduced in , where both allow secure evaluation of functions represented as Boolean circuits. Although powerful with respect to the functions they can evaluate, garbled circuits require constructing high depth circuits and transmitting them over a communication channel, which incurs a substantial cost in computation and communication. While many improvements in the efficiency of garbled circuits have been made over the years, the complexity still prohibits the usage of such schemes in real time applications .
Another path within secure multiparty computation with computational security guarantees is computing on encrypted data. This approach, suggested in  under the name “privacy homomorphisms”, is currently known as homomorphic encryption. Homomorphic encryption allows evaluating specific types of functionals on encrypted inputs. Homomorphic encryption comes in three flavors: fully, somewhat and partially homomorphic encryption. Fully homomorphic encryption allows evaluation of Boolean functions over encrypted data and was introduced in , and further improved in terms of efficiency in [9, 10, 11]. However, the computational overhead is prohibitive, due to the complexity of the cryptosystem primitives and size of the encrypted messages . Somewhat homomorphic encryption schemes, e.g. , are less complex, but support only a limited number of operations (additions and multiplications). Partially homomorphic encryption schemes are tractable but can support either multiplications between encrypted data, such as El Gamal , unpadded RSA  or additions between encrypted data, such as Paillier , Goldwasser-Micali , DGK .
While the above techniques ensure that the output is correctly computed without revealing anything about the inputs in the process, depending on the application, an adversary might be able to learn information about the input only from the output. Techniques such as differential privacy ensure that the revealed output does not break the privacy of the input. Differential privacy, introduced in  and surveyed in [19, 20], adds controlled randomness in the data, so that the results of statistical queries do not tamper with the privacy of the individual entries in a database, by introducing uncertainty in the output. However, the noise determines a trade-off between the accuracy of the results of the differentially private queries and the privacy parameter.
I-a Related work
Homomorphic encryption has been recently used to design encrypted controllers, with partially homomorphic encryption schemes for linear controllers in [21, 22], model predictive control applications in [23, 24], with fully homomorphic encryption in , and to design observers in 
. Unconstrained optimization problems are commonly used in machine learning applications, e.g., for linear regression, and several works have addressed gradient methods with partially homomorphic encryption[27, 28] or with ADMM in a distributed setting . However, adding constraints substantially complicates the optimization problem by restricting the feasible domain. In addition, constraints introduce nonlinear operations in the optimization algorithm, which cannot be handled inherently by the partially homomorphic cryptosystems. In our prior work [30, 31], we showed how to overcome these difficulties by using blinded communication in a centralized setup. Recently,  proposed a distributed approach to the constrained optimization problem with homomorphic encryption, where each party locally performs the projection on the feasible domain on unencrypted data.
Examples of works that tackle private linear programming problems or machine learning applications with optimization subroutines using garbled circuits, secret sharing, and hybrid approaches between the above are[33, 34, 35, 36, 37] and the references within. In control applications, works such as [38, 39, 40, 41] employ differential privacy to protect dynamic data. Furthermore, examples such as [42, 43, 44] make use of differential privacy techniques in optimization algorithms. Other lines of work solve optimization problems with non-cryptographic methods: using transformations, e.g. [45, 46, 47, 48], which guarantee an uncertainty set around the private data, or using arbitrary functions for obfuscating the objective functions in a special graph topology .
In this paper we consider a privacy-preserving cloud-based constrained quadratic optimization. In control theoretic applications and machine learning, linear and quadratic optimization problems arise frequently – e.g., state estimation under minimum square error, model predictive control, support vector machines – which require privacy guarantees of the data involved in the computation.
We develop a new tractable optimization protocol to privately solve constrained quadratic problems. Our protocol relies on cryptographic tools called encryption schemes. To solve the optimization problem on encrypted data, we use an additively homomorphic encryption scheme, where, in short, addition commutes with the encryption function. Thus, a party can process encrypted data without having direct access to the data. The novelty is how to handle in a privacy-preserving manner the constraints of the problem that introduce non-linearities that cannot be supported by additively homomorphic encryption schemes. We show that a projected gradient method that operates on the Lagrange dual problem can alleviate this problem and can be efficiently run on encrypted data by exploiting communication between the participating parties.
The main contributions of our work are the following:
[topsep=0pt,wide, labelwidth=!, labelindent=0pt]
We formally state and prove computational security guarantees for such protocol. The proof relies on applying cryptographic tools to the specific optimization algorithm that runs over multiple iterations.
We propose an alternative protocol which sacrifices some privacy but involves less communication overhead.
We implement the protocols and show the computational and communication complexity produce reasonable running times.
Furthermore, we emphasize the difference between the computational security guarantee with the non-unique retrieval guarantee that is important in such optimization problems. We finally point out that the present manuscript provides detailed security proofs and analyses not available in previous versions of the papers [30, 31].
Organization. The rest of the paper is organized as follows: we formulate the problem in Section II and formally introduce the computational security guarantees in Section III. We describe the cryptographic tools we use in Section IV, more specifically, the partially homomorphic scheme, along with its properties and the symmetric encryption used. In Section V, we describe the optimization theory behind the proposed protocol and justify the choice of the gradient method. Furthermore, we present the main protocol and the subroutines that compose it and in Section VI, we show that the protocol achieves privacy of the agent’s data and of the target node’s result. We discuss possible relaxations of the security guarantees and propose a more efficient protocol under these weaker conditions in Section VII. In Section VIII, we provide a privacy analysis of the problem concerning the input-output relation. We present the complexity analysis of the protocols and show that the experimental results point out reasonable running times in Section IX. Additional tools necessary for proving the security of the proposed protocol are introduced in Appendix A and further details on the protocols used are given in Appendix B. Finally, the detailed privacy proofs are given in Appendix C.
Notation. We denote matrices by capital letters. represents the set of matrices with real elements and represents the set of symmetric positive definite matrices. Element-wise inequalities between vectors are denoted by . The notation means that is uniformly drawn from a distribution . Pr
denotes the probability taken over a specified distribution. We refer to algorithms that are run interactively by multiple parties as protocols.
Ii Problem Setup
Ii-a Motivating Examples
Quadratic optimization is a class of problems frequently employed in control system design and operation. As a first motivating example, consider the problem of estimating the state of a dynamical system from privacy-sensitive sensor measurements. For instance, such a problem arises in smart houses where the temperature or energy readings of the sensors are aggregated by a cloud controller and can reveal whether the owners are in the building. In particular, let the system dynamics and sensor measurements be described by:
for , where are process and measurement noise respectively. The system and sensor parameters can be thought as publicly available information while the sensor measurements are privacy-sensitive data. The untrusted cloud has to collect the measurements and output an initial state estimate of the system to a target agent, while maintaining the privacy of the sensor data and final output. A simple state estimate may be found as the solution to the least squares problem:
and is the well-known system observability matrix. More general state estimation problems may also include constraints, e.g., the initial state may lie in a polyhedral set where the shape of the polyhedron captured by the matrix is publicly known but its position and size captured by the vector is private information.
As a second example, consider a control problem with privacy-sensitive objectives. Suppose we are interested in steering a dynamical system:
starting from a private initial position while guaranteeing safety state constraints of the form:
where for are private values. Such problems arise when exploring vehicles are deployed in uncertain or hazardous environments or when different users compete against each other and want to hide their tactics.
Denote by and the private values such that the system is guaranteed to track a reference . Then, we need to solve the problem:
where are positive definite cost matrices and we used the quadratic norm notation .
Ii-B Problem statement
The above examples can be modeled as constrained quadratic optimization problems with distributed private data. We consider three types of parties involved in the problem: a number of agents , , a cloud server and a target node . The purpose of this setup is to solve an optimization problem with the data provided by the agents and the computation performed on the cloud and send the result to the target node. The architecture is presented in Figure 1.
Let us consider a strictly-convex quadratic optimization problem, which we assume to be feasible:
where the variables and the parties to which they belong to are described as follows:
Agents : The agents are parties with low computational capabilities that possess the private information and . The private information is decomposed across the agents as: and , with and being the private data of agent such that and .
Cloud : The cloud is a party with high computational capabilities that has access to the matrices and . When the computation is sophisticated and/or involves proprietary algorithms, and are private data of the cloud. In order to capture a greater number of problems, we will also consider the case where or are public.
Target Node : The target node is a party with more computational capabilities than the agents that is interested in the optimal solution of the problem. The target node can be either one of the agents or a separate cloud server.
Note that in the first motivating example (2), the matrix in (8) corresponds to the publicly known matrix and the vector corresponds to the private vector . For the objective to be strongly convex, we require that , which also implies standard system observability. Similarly, in the second example (7), the matrix is composed from the regulating cost matrices, the cost vector is formed from the private initial conditions and steady-state solution for the reference tracking, mixed by the system’s dynamics, and the constraints vector depends on the private state bounds and initial condition.
Ii-C Adversarial model
In most cloud applications, the service provider has to deliver the contracted service or otherwise the clients would switch to another service provider. This means that the cloud cannot alter the data it receives. Moreover, the clients’ interest is to obtain the correct result from the service they pay for, hence, the agents and target node will also not alter the data. However, the parties can locally process the data they receive in any fashion they want. This model is known as semi-honest, which is defined as follows:
(Semi-honest model) A party is semi-honest if it does not deviate from the steps of the protocol but may store the transcript of the messages exchanged and process the data received in order to learn more information than stipulated by the protocol.
This model also holds when considering eavesdroppers on the communication channels. An adversarial model that only considers eavesdroppers as adversaries differs from the semi-honest model by the fact that, apart from the untrusted channels, the parties that perform the computations are also not trusted. Malicious and active adversaries – that diverge from the protocols or tamper with the messages – are not considered in this paper.
The purpose of the paper is to solve Problem (8) using a secure multiparty computation protocol for semi-honest parties. This protocol takes as inputs the private data of the agents, as well as the cloud’s data, and involves the parties in exchanging messages and participating in some specified computations, and eventually outputs to the target node the solution of the optimization problem. This protocol should guarantee computational security. More specifically, the cloud cannot obtain any information about the private inputs of the agents and the output of the target node and similarly, the target node cannot obtain any information about the private inputs of the agents and the cloud, other than what they can compute using their inputs and outputs and public information, by running a computationally efficient algorithm after the execution of the protocol.
Iii Privacy goals
We introduce some notions necessary to the security definitions. In what follows, defines a sequence of bits of unspecified length. An ensemble
is a sequence of random variables ranging over strings of bits of length polynomial in, denoted by , arising from distributions defined over a finite set .
(Statistical indistinguishability [50, Ch. 3]) The ensembles and are statistically indistinguishable, denoted , if for every positive polynomial , and all sufficiently large :
where the quantity on the left is called the statistical distance between the two ensembles.
It can be proved that two ensembles are statistically indistinguishable if no algorithm can distinguish between them. Computational indistinguishability is a weaker notion of the statistical version, as follows:
(Computational Indistinguishability [50, Ch. 3]) The ensembles and are computationally indistinguishable, denoted , if for every probabilistic polynomial-time algorithm , called the distinguisher, every positive polynomial , and all sufficiently large , the following holds:
The previous definition is relevant when defining the privacy goals to be guaranteed by the protocol: two-party privacy of the sensitive data of the parties. The intuition is that a protocol privately computes a functionality if nothing is learned after its execution, i.e., if all information obtained by a party after the execution of the protocol (while also keeping a record of the intermediate computations) can be obtained only from the inputs and outputs available to that party.
(Two-party privacy w.r.t. semi-honest behavior [51, Ch. 7]) Let be a functionality, and , denote the first and second components of , for any inputs . Let be a two-party protocol for computing . The view of the -th party () during an execution of on the inputs , denoted , is , where represents the outcome of the ’th party’s internal coin tosses, and represents the -th message it has received. For a deterministic functionality , we say that privately computes if there exist probabilistic polynomial-time algorithms, called simulators, denoted , such that:
This definition assumes the correctness of the protocol, i.e., the probability that the output of the parties is not equal to the result of the functionality applied to the inputs is negligible .
For protocols that involve more than two parties, Definition 4 can be extended to multi-party privacy, by also taking into consideration coalitions of semi-honest parties. We present this extension in Definition A.1, along with further underlying details related to the cryptographic notion of security and its interpretation in Appendix A-A.
Iv Encryption scheme
Iv-a Additively Homomorphic Encryption
Let define a generic encryption primitive, with domain the space of private data, called plaintexts, and codomain the space of encrypted data, called ciphertexts. For probabilistic encryption schemes, the encryption primitive also takes as input a random number. The decryption primitive is defined on the space of ciphertexts and takes values in the space of plaintexts. Additively homomorphic schemes satisfy the property that there exists an operator defined on the space of ciphertexts such that:
for any plaintexts supported by the scheme. Here, we use set inclusion instead of equality because the encryption of a message is not unique in probabilistic cryptosystems. Intuitively, equation (9) means that performing this operation on the two encrypted messages, we obtain a ciphertext that is equivalent to the encryption of the sum of the two plaintexts. Formally, the decryption primitive is a homomorphism between the group of ciphertexts with the operator and the group of plaintexts with addition , which justifies the name of the scheme. It is immediate to see that if a scheme supports addition between encrypted messages, it will also support subtraction, by adding the additive inverse, and multiplication between an integer plaintext and an encrypted message, obtained by adding the encrypted messages for the corresponding number of times.
In this paper, we use the popular Paillier encryption, but any other additively homomorphic encryption scheme can be employed. The Paillier cryptosystem  is an asymmetric additively homomorphic encryption scheme. Asymmetric or public key cryptosystems involve a pair of keys: a public key that is disseminated publicly, and which is used for the encryption of the private messages, and a private key which is known only to its owner, used for the decryption of the encrypted messages. In this section, we will only present the notation for the encryption and for the allowed operations between the encrypted messages and postpone the implementation details to Appendix A-B.
In the Paillier encryption scheme, the private data are elements of the ring of integers modulo , denoted by , where is a large integer of bits, called the Paillier modulus. The encrypted messages take values in the positive integers modulo , denoted by . Paillier is a probabilistic encryption scheme, which means that the encryption primitive also takes as an argument a random number which, for simplicity, we avoid in this notation. For a plaintext message , we denote the Paillier encryption by , for an instance of the random number. For readability, we will use throughout the paper the following abstract notation for the operations on the encrypted space:
where means that the equality holds after applying the decryption primitive on both sides . The expressions for the abstracted operations are described in Appendix A-B. We will slightly abuse these notations to denote encryptions of vectors and additions and multiplication by vectors and matrices. In our protocol, the target node is the owner of a pair of Paillier keys. Hence, for everything that follows, denotes the encryption with the target node’s public key .
Example: Private unconstrained quadratic optimization. In order to illustrate the usage of the Paillier encryption scheme for computations on encrypted data, consider the following example of an unconstrained optimization problem:
where the variables have the same meaning as in Problem (8) described in Section II. The cloud has access to the matrix . If the cloud also had access to private data from the agents, the gradient method could be employed to obtain the optimal unencrypted solution of Problem (11). An iteration of the gradient method has the form:
where is the step-size chosen by the cloud and for a number of iterations . However, in order to protect their private data and the privacy of the solution , the agents send the value encrypted to the cloud as . It is very important to notice that for a quadratic objective function, only linear operations in the private data and are required in order to compute one iteration of the gradient descent algorithm. Hence, by taking advantage of the additively homomorphic property (10), the cloud can locally compute the optimal solution by the gradient descent in the encrypted domain, for all :
and send the final point to the target node to decrypt and obtain the desired result. Such a protocol satisfies the desired security according to Definition 4 provided in Section III by the fact that the cloud only has access to data encrypted by a semantically secure encryption scheme, such as Paillier . More details are provided in Section VI.
However, the optimization problem considered in this paper is a quadratic problem with constraints, which introduce nonlinear operations in the optimization algorithm and cannot be handled inherently by the Paillier cryptosystem. We will describe in Section V how we leverage communication between the parties in order to achieve privacy-preserving nonlinear operations.
Iv-B Symmetric encryption scheme
Apart from the (public key) partially homomorphic cryptosystem above, we will also use a specific symmetric key cryptosystem for reasons that will become apparent in the protocol, related to hiding the data from the target node. This symmetric key cryptosystem is similar to the well-known one-time pad[53, 54], and can be thought of as additively blinding a private value by noise. In contrast to public key cryptography, symmetric key algorithms perform the encryption and decryption with the same key. For messages of bits, the key is generated as with length of bits, where is the security parameter. The encryption primitive is , with , and the decryption is obtained as . This is also called two-out-of-two additive secret sharing  when the addition is on an abelian group. The security of this scheme lies on generating a uniformly random key and on using this key for encryption only once, which yields that the distribution of is statistically indistinguishable from a random number sampled of bits. For this reason, is called the statistical security parameter. We also notice that this symmetric cryptosystem is compatible with the Paillier cryptosystem, in the sense that the two encryptions commute: by using instead of for decryption, where is performed on the message space of the Paillier cryptosystem. Such a scheme is commonly employed for blinding messages, for instance in [56, 57, 58].
For simplicity, in the protocols described in Section V, we will directly use the summation with a random number to depict the above symmetric key cryptosystem, respectively the difference by the same random number for decryption.
The strength of a cryptosystem relies on the computational intractability of retrieving the private key from the public information – an adversary holding the public information cannot find the private key by brute force computations. Public key cryptosystems like the Paillier cryptosystem involve a security parameter , which is usually the length of the modulus of the public key, and symmetric key cryptosystems involve a security parameter , which is the bit-length of the key. As is common in practice, increasing the size of the security parameters to values larger than, say 512 bits for and 80 bits for , increases the security of the system against any computationally efficient inference attack.
V Secure Constrained Quadratic Optimization
In this section we introduce our main algorithmic results. Let us first describe the optimization algorithm used for solving the minimization problem (8) on unencrypted data.
For strongly convex problems, one can resort to duality theory [59, Ch. 5] to compute the projection on the feasible set, and be able to retrieve the optimal value of the primal problem from the optimal value of the dual problem. For the quadratic optimization problem (8), its dual is also a quadratic optimization problem:
The dual objective function is denoted by and its gradient is equal to:
Under standard constraint qualifications, e.g., Slater’s condition [59, Ch. 5], strong duality between the primal and dual holds, which means the optimal objective in the primal problem (8) is equal to the objective in the dual problem (13). Moreover, the optimality conditions (Karush-Kuhn-Tucker) hold and are the following:
For strictly convex problems, i.e., , the optimal solution of the primal problem can be obtained from (15) as
An algorithm for computing the optimum in problem (13), which we will show is also compatible with the partially homomorphic encryption of Section IV, is the projected gradient ascent method. The projected gradient ascent is composed by iterations of the following type:
where is the step size and is the projected value of over the non-negative orthant. For full rank of , the dual problem is strictly convex and the algorithm converges with a linear rate  for a fixed step size , where . For non-strictly convex dual function, the gradient ascent algorithm converges in sublinear time .
V-a Projected gradient ascent on encrypted data
As stated in Section II, we aim to solve an optimization problem outsourced to the cloud on private distributed data from the agents and send the result to the target node. To protect the agents’ data, we use an encryption scheme that allows the cloud to perform linear manipulations on encrypted data, as described in Section IV. To this end, the target node generates a pair of keys and distributes the public key to the agents and the cloud, enabling them to encrypt their data, which only the target node will be able to decrypt, using the private key. We consider that all the data is represented on integers of bits and comment on this further in Section V-E.
The main difficulty in performing the projected gradient ascent on encrypted data is performing iteration (18). We have already seen in the example in Section IV-A that the update of the iterate in the direction of the gradient ascent can be computed locally by the cloud directly on the encrypted data (12). However a first challenge lies in performing the comparison with zero. Due to the probabilistic nature of the Paillier encryption scheme, the order on the plaintext space is not preserved on the ciphertext space and comparison on encrypted data cannot be performed locally by the cloud. Notwithstanding, we employ an interactive protocol between the cloud, which holds the encrypted data, and the target node, which is the owner of the private key for the Paillier cryptosystem, that achieves the comparison securely. Moreover, after the comparison is performed, the update of the encrypted iterate (18) has to be done in a private way, so that the result of the maximum operation is not revealed to any of the parties involved (the cloud and the target node). These two steps are the main computational bottleneck in the protocol we propose, as both require secure communication between the cloud and the target node. A preliminary version of this solution was presented in .
We can privately achieve the two steps mentioned above in three stages. First, the cloud has to randomize the order of the two encrypted variables it wants to compare (Protocol V-B). Second, the cloud and target engage in an interactive comparison protocol that takes as inputs the two randomized variables and outputs the result of the comparison to the target node (Protocol V-B). Third, the update of the dual iterate is achieved through an interactive protocol between the cloud and target node, which takes as inputs the two randomized variables and the result of the comparison and outputs to the cloud the updated iterate (Protocol V-C). Throughout this paper, by comparison we mean element-wise comparison, since the variable is a vector.
V-B Secure comparison protocol
In order to privately compute (18), i.e., hide the result from all the parties involved, we want to keep the result of the comparison of the updated iterate with zero unknown to both the cloud and the target node. The comparison protocol will reveal the result of the comparison between the two inputs to the target node . However, if we introduce an additional step where randomizes the order of the two values that it wants to compare, then does not learn any information by knowing the result of the comparison.
Protocol 1: Randomization step
Next, we will demonstrate how the comparison protocol works via the following instantiation. Damgård, Geisler and Krøigaard introduced a protocol in [17, 61] for secure comparison of two private inputs of different parties, which we will call the DGK protocol. To this end, they also propose an additively homomorphic encryption scheme, which has the property that checking if the value zero is encrypted is efficient, which is useful for comparisons and working with bits. An extension of this protocol to the case where none of the parties knows the two numbers that have to be compared, which is of interest to us, and some improvements in terms of efficiency were proposed in .
The comparison protocol that will be used in our optimization protocol is as follows. Let have two encrypted values under the Paillier scheme and that it obtained after running Protocol V-B, and let have the decryption key. Furthermore, let also have the decryption key of the DGK homomorphic encryption scheme, which we describe in Appendix B. At the end of the protocol, will have the result of the comparison in the form of one bit such that . Let denote the number of bits of the unencrypted inputs . Protocol V-B is based on the fact that the most significant bit of is the bit that indicates if . As already explained in Section II, we will use blinding by random numbers to secure the communication from the cloud to the target node.
V-C Secure update protocol
Moreover, we need to ensure that when the cloud updates the value of the dual iterate at iteration in equation (18), it does not know the new value. The solution is to make the cloud blind the values of and and send them to the target node in this order, where the latter selects the value accordingly to the comparison result and then sends it back to the cloud. However, there are two important issues that have to be addressed in order for the update step to not leak information about the sign of the iterate: the blinding should be additive and effectuated with different random values, and the ciphertexts should be refreshed. The reasons are the following: if the blinding is multiplicative, by decrypting the product, the target knows which one of the values is zero. Moreover, if the two values are additively blinded with the same random value, the target can subtract them and reveal at least if the value is zero. Re-randomization of the encryptions is necessary so that the cloud cannot simply compare and with the received value. This can be done by adding an encryption of zero or by decryption followed by encryption. Protocol V-C is the solution to the update problem:
Protocol 3: Secure update of the dual variable
The intuition behind the protocol is that if , then and we obtain , and otherwise, and we obtain . In both cases the cloud correctly updates the dual variable with the projected value.
V-D Protocol for solving strictly-convex quadratic problems
Having defined these protocols, we can now build a protocol that represents one iteration (18) of the dual projected gradient ascent method.
Protocol 4: Secure iteration of the dual projected gradient ascent method
Line 3 ensures that the updated iterate has the required number of bits for the comparison protocol. This step is achieved by an exchange between the cloud and target node: the cloud additively blinds the iterate by a random number, sends it to the target node, which decrypts and truncates the sum and sends it back, where the cloud then subtracts the truncated random number.
Using the building blocks described above, we can finally assemble the protocol that privately solves the constrained quadratic optimization problem (8) with private data and sends the optimal solution to the target node. The public key and bit-length are known by all the parties, hence we omit them from the inputs.
Protocol 5: Privacy preserving algorithm for solving strictly-convex quadratic optimization problems
V-E Fixed-point arithmetic
The optimization problem (8) is defined on real variables, whereas the Paillier encryption scheme is defined on integers. To address this issue, we adopt a fixed-point arithmetic setting, where we allow for a number to have a fixed number of fractional bits. First, we consider numbers that have the magnitude between . Second, we consider a value having bits for the integer part and bits for the fractional part. Therefore, by multiplying the real values by and truncating the result, we obtain integers. We choose large enough such that the loss in accuracy is negligible and assume that there is no overflow. For ease of exposition, we consider this data processing done implicitly in the protocols described.
The random numbers used for blinding the sensitive values (in Protocols V-B and V-C) are sampled uniformly from the integers in of bits, where is the statistical security parameter, as already explained in Section II, chosen such that brute-forcing the solution is intractable. In order to guarantee correctness of the comparison protocol, no overflow must take place, so we must impose .
The errors in the solution caused by the fixed-point arithmetic operations necessary for the encryption can be analyzed with the same tools as in [63, 64, 24]. The round-off errors can be regarded as states in a stable dynamical system with bounded disturbances, and hence, have a bounded norm that offers a guide on how to choose the number of fractional bits for the fixed-point representation. On the other hand, the overflow and quantization errors depend on the magnitude of the dual iterates. We considered feasible and bounded problems – the dual problem (13) has a finite solution – therefore, one can select the number of integer bits in the representation such that no overflow occurs.
Vi Privacy of Quadratic Optimization Protocol
We will now introduce the main theoretical results of the paper. In the interest of completeness of the paper, more theoretical preliminaries needed for the privacy results, such as semantic security, are given in Appendix A. We provide the statements of the results and the main ideas of the proofs here, and include the detailed arguments in Appendix C.
The intuition for the proof is as follows. Consider an iteration of the gradient ascent in Protocol 2. Firstly, in the Paillier cryptosystem, two ciphertexts are computationally indistinguishable to a party that does not have access to the decryption key. Secondly, the exchanges between the cloud and the target are additively blinded using a different random number uniformly sampled from a large enough range (at least bits more over the values that are desired to be blinded, where the size of is chosen appropriately, as discussed in Section IV-B. This means that the blinded message is statistically indistinguishable from a random number sampled from the same distribution. Thirdly, the ciphertexts are refreshed (a different encryption of the same value), after each exchange, so a party that does not have access to the decryption key cannot infer information about the encrypted values by simply comparing the ciphertexts. Then, none of the parties can infer the magnitude or the sign of the private variables. However, we need to show that privacy is not broken by running an iteration multiple times. We prove that storing the exchanged messages does not give any new information on the private data, using similar arguments. Formally, using Definition 4, we construct a probabilistic polynomial-time simulator that randomly generates messages from the inputs and outputs such that its view and the view of the adversary, on the same inputs, are computationally indistinguishable. The correctness of Protocol 2 is immediate and follows from the correctness of the dual gradient ascent algorithm and the correctness of the comparison protocol. The detailed proof is given in Appendix C-A.
Let us now consider collusions between the parties in the setup and prove privacy of Protocol 2 under coalitions. The definition of privacy is naturally extended in this case and we can further establish the computational security in the following result – see Appendix A for more details. Definition A.1 states that even under collusions, the protocol securely computes a functionality, which in this case, is solving a quadratic optimization problem. No further information is revealed than what can inferred from the coalition’s inputs and outputs. However, if all agents and the cloud collude then they have access to all the information to solve the problem, in which case the above result is rather vacuous. Similarly, if the cloud colludes with the target node, then it can gain access to all the private data of the agents. Hence, we consider coalitions that involve either all the agents or the cloud and the target node to be prohibited (and unrealistic) and only consider coalitions between a strict subset of the agents and the cloud or between a strict subset of the agents and the target node. We give a complementary analysis on the information gained from the inputs and outputs of coalitions in Section VIII.
Vii Alternative quadratic optimization protocol without secure comparison
As explained in Section V-B, the major computation and communication overhead of the above protocol is the secure comparison protocol, required to project dual variables to non-negative numbers. In this section, we describe a computationally less involved alternative approach, which bypasses the need for the secure comparison protocol, at the expense of revealing more information. This approach is developed in more detail in previous work .
Specifically, consider a step of the dual gradient algorithm where the cloud maintains an unprojected gradient ascent step encrypted using the public key of the target node. Suppose the cloud multiplies the elements in this vector with random scalar values uniformly distributed over the positive integers and sends the products to the target node. The latter can decrypt the message using its private key and gain access not to the actual unprojected iterate but to the randomly scaled version of it, which reveals the sign. It can then project it to the non-negative orthant in an unencrypted fashion for , and finally encrypt the result using its own public key and return it to the cloud. The cloud can divide the result with the previously selected values to compute in an encrypted fashion the elements which is equivalent to the encrypted actual projected step for , because division with a positive number commutes with the max operator.
This protocol is presented next, and it can be employed as an alternative for the more demanding Protocol V-D, as it bypasses the complexity of the secure comparison part. On the other hand, as can be seen from the outputs of the protocol, it reveals more information to the target node than the secure comparison approach. In particular it reveals a scaled version of the unprojected dual variable which in turn does not reveal the magnitude of this value but reveals whether this is positive, negative, or zero.
Protocol 6: Alternative iteration of the dual projected gradient ascent method
Recall that we can only perform operations on integers when using the Paillier cryptosystem. To this end, we performed an implicit multiplication by and truncation when encrypting numbers and a division by the corresponding factor when decrypting. For Algorithm VII to work, we slightly modify this technique. The reasoning is the following: we want the values of to be large enough to blind the magnitude of the elements of , so we will randomly sample from the values of bits, where is the security size for the multiplicative blinding. Furthermore, we need to be able to represent on the message space and to have enough precision as to obtain the correct value of in line 7. Hence, we will perform implicit multiplications and divisions by , for . This change shows that, although significantly less communication is required to perform the projection operation, all the other operations are more expensive, because we work with larger numbers.
Viii Privacy discussion
In this section we discuss privacy aspects that differ from the computational notions of Definitions 4 and A.1. Even if a protocol does not leak any information, the known information in a coalition (e.g., the output and some of the inputs) can be used by the coalition to infer the rest of the private inputs. More specifically, in our optimization problem, the private variables and the optimal solution are coupled via the optimality conditions (15)-(17), which are public knowledge, irrespective of the protocol, and may be used for the purpose described above.
Consider the following definition that concerns the retrieval of private data from adversarial/known data.
(Non-unique retrieval) Let be the private inputs of a problem and let an algorithm solve that problem. Let be the adversarial knowledge of the problem, which can contain public information, some private information (including some parts of the input) and the output of algorithm for the adversary, denoted by . We say cannot be uniquely retrieved by the adversary if there exists a set , such that , and:
Definition 5 can be modified to be stronger by requiring the set to have an infinite number of elements.
In what follows, beyond the computational security analysis of the previous sections, we carry out an algebraic analysis on a black-box protocol that given the agent’s private data and the cloud’s matrices , outputs the solution of Problem (8) to the target node. We provide conditions such that a coalition cannot uniquely determine unknown private inputs from the output and a set of inputs, in the sense of Definition 5. In particular, this analysis applies to Protocol 2 which, assuming it runs for sufficient iterations, outputs the desired result to the target node. We perform this algebraic analysis in the space of real numbers, which can be further expanded to fixed-point arithmetics for large enough precision.
Suppose without loss of generality that a coalition between agents () has access to the elements with , and with . Then let us define the decomposition of the matrix as:
where the matrices , , .
Consider a protocol solving Problem (8) and a coalition between the target node and agents with access to of the values of and of the values of . Suppose the cost and constraint matrices , are public. Then:
if and there exists a vector such that and , then the coalition cannot uniquely retrieve the value of ;
if additionally and then the coalition cannot uniquely retrieve the value of .
The proof is based on the fact that the variables , , , satisfy the optimality conditions (15)-(17). Specifically, these are conditions that the unknown variables and , as well as the optimal dual variables , must satisfy given all the rest of the known variables. Hence, if there are multiple such solutions to the KKT conditions the coalition cannot uniquely determine the private variables. The proof is given in Appendix C-D.
Consider a protocol solving Problem (8) and a coalition between the cloud and agents with access to of the values of and of the values of . Then, a coalition that satisfies and cannot uniquely retrieve the values of and .
The proof easily follows from the fact that is a positive definite matrix, hence, the solution is finite, and does not have any columns or rows of zeros. A coalition between the cloud and the agents cannot solve (8) since it lacks all the data to define it ( and ), so it cannot uniquely retrieve and the rest of the agents’ private data.
Propositions 3 and 4 give general sufficient conditions for the desired non-unique retrieval property to hold and are independent of the exact problem instance values. If the conditions stated are not satisfied, then the previous result in Theorem 2 still holds. However, the additional non-unique retrieval property may fail because the inputs and outputs of the coalition are sufficient to leak information about the rest of the private inputs. The above analysis can also be extended to the case where some of the matrices are private information.
Let us now use Definition 5 to analyze Protocol VII and the effect the release of more information to the target node has on the privacy of the inputs of the honest agents. We perform the analysis at a setup where the protocol runs for a sufficient number of iterations and hence the dual variable has converged to the true optimal value and the algorithm has also converged to the true optimal primal value . Suppose the target node has access to the sign of the unprojected optimal dual variable – note that this is the case when we employ Protocol VII. In combination with the solution , this information can be further employed by the target node to infer private information of the agents.
Consider a protocol solving Problem (8) where the target node has access to the solution and also the sign of the unprojected optimal dual variables . Suppose further the matrix is publicly available. Then the private values cannot be uniquely retrieved by the target node if and only if for some .
Here implies that the corresponding optimal dual value is zero, . This means that the corresponding constraint in problem (8) is inactive at the optimal solution [59, Ch. 5] and the target node cannot uniquely determine the th element of the corresponding bound . In the opposite case, if all constraints (16) are either active or redundant at the optimal solution (), this is revealed to the target node because in that case , and the private value is uniquely determined by .
Consider the setup of Proposition 5 and the matrix is publicly available. The private values cannot be uniquely retrieved from the outputs and of the target node if and only if for some .
The case corresponds now to the case where the corresponding constraint is active at the optimal solution . When this fails, all constraints are inactive at and they do not play a role. Hence, we have an unconstrained quadratic problem in (8), i.e., the optimal solution satisfies the first order condition , which reveals the value of to the target. To guarantee privacy with respect to both private values we need both an inactive constraint () as well as an active constraint () for some , leaving some uncertainty in the estimations performed by the target node. Similar analysis may be performed for collusions, e.g., of the target node and some agents.
Finally, we note that here we analyzed the non-unique retrieval properties of the problem considering black-box protocols. It is of value to also consider additional steps when designing a protocol that solves Problem (8) to prevent information leakage if the conditions in the propositions are not satisfied. It may be possible to use techniques such as Differential Privacy to augment the cryptographic privacy of the protocol with output privacy, by adding correlated data to the inputs. This approach lies out of the scope of our paper, but works such as [42, 43, 44] may be helpful to this end.
The efficiency of a secure multi-party protocol is measured in the complexity of the computations performed by each party, along with the rounds of communications between the parties. While the former is relevant from the perspective of the level of computational power required, the latter are relevant when communication is slow.
In the setup we considered, the agents are practically low-power platforms, hence, they are only required to effectuate one encryption and one communication round, but the cloud and the target node are platforms with high computational capabilities, e.g., servers.
Let be the bit-length of the modulus . Then, the size of a Paillier ciphertext is , regardless of the size of the plaintext in . A Paillier encryption takes one online exponentiation and one multiplication modulo , and a decryption takes one exponentiation modulo and one multiplication modulo . The addition of two ciphertexts takes one modular multiplication modulo . A multiplication of a ciphertext with a plaintext of bits is achieved as a modular exponentiation modulo . A multiplication of two elements in can be achieved in . A modular exponentiation in with an -bit exponent can be computed in and can be sped up via the Chinese Remainder Theorem when the factorization of is known. A DGK encryption takes one modular exponentiation and one modular multiplication in , and a DGK decryption – checking if the encrypted message is 0 or not – takes an exponentiation modulo .
We implemented the protocol proposed in Section V-D in Python 3 and ran it on a 2.2 GHz Intel Core i7 processor. For random instances of the data in Problem (8), with and bits, and 0 ms delay for communication, we obtain the average running times depicted with the dashed gray lines in Figure 2 and Figure 3. The simulations are run for 30 iterations, for ease of comparison.
Figure 2 depicts the online execution time of Protocol 2 with the iterations as in Protocol V-D. As expected, because we work in the dual space, the time required for running the iterations varies very little with the number of variables, but increases with the number of constraints. However, in the case of Figure 3, which depicts the online execution time of Protocol 2 with the iterations as in Protocol VII, the time varies more with the number of variables and not as much with as Protocol V-D does. The reason is that we work with substantially larger numbers than in the previous case, due to the large factor with which we have to multiply in order to be able to de-mask the dual iterate, which amplifies the time difference between problems of different dimensions.
The trade-off between the privacy and communication can be seen when we artificially add a communication delay of 2 ms between the cloud and the target node, to simulate the delays that can occur on a communication network. It can be observed in Figures 2 and 3 that the communication delay has a smaller effect on the less private Protocol VII than on the fully private Protocol V-D.
This work was partially sponsored by the NSF awards 1739816, 1705135, CNS-1505799 and the Intel-NSF Partnership for Cyber-Physical Security and Privacy, ONR N00014-17-1-2012, and by TerraSwarm, one of six centers of STARnet, a Semiconductor Research Corporation program sponsored by MARCO and DARPA. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of NSF, or the U.S. Government. Andreea Alexandru would like to thank Bret Hemenway for helpful discussions.
-  S. Subashini and V. Kavitha, “A survey on security issues in service delivery models of cloud computing,” Journal of Network and Computer Applications, vol. 34, no. 1, pp. 1–11, 2011.
-  D. A. Fernandes, L. F. Soares, J. V. Gomes, M. M. Freire, and P. R. Inácio, “Security issues in cloud environments: a survey,” International Journal of Information Security, vol. 13, no. 2, pp. 113–170, 2014.
-  “List of data breaches,” https://en.wikipedia.org/wiki/List_of_data_breaches.
-  A. C. Yao, “Protocols for secure computations,” in Proc. 23rd Symposium on Foundations of Computer Science. IEEE, 1982, pp. 160–164.
-  S. Goldwasser, S. Micali, and A. Wigderson, “How to play any mental game, or a completeness theorem for protocols with an honest majority,” in Proc. 19th ACM STOC, vol. 87, 1987, pp. 218–229.
-  M. Bellare, V. T. Hoang, and P. Rogaway, “Foundations of garbled circuits,” in Proc. Conference on Computer and Communications Security. ACM, 2012, pp. 784–796.
-  R. L. Rivest, L. Adleman, and M. L. Dertouzos, “On data banks and privacy homomorphisms,”