Outsourcing Private Machine Learning via Lightweight Secure Arithmetic Computation

In several settings of practical interest, two parties seek to collaboratively perform inference on their private data using a public machine learning model. For instance, several hospitals might wish to share patient medical records for enhanced diagnostics and disease prediction, but may not be able to share data in the clear because of privacy concerns. In this work, we propose an actively secure protocol for outsourcing secure and private machine learning computations. Recent works on the problem have mainly focused on passively secure protocols, whose security holds against passive (`semi-honest') parties but may completely break down in the presence of active (`malicious') parties who can deviate from the protocol. Secure neural networks based classification algorithms can be seen as an instantiation of an arithmetic computation over integers. We showcase the efficiency of our protocol by applying it to real-world instances of arithmetized neural network computations, including a network trained to perform collaborative disease prediction.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

09/02/2021

CrypTen: Secure Multi-Party Computation Meets Machine Learning

Secure multi-party computation (MPC) allows parties to perform computati...
02/06/2021

Privacy-Preserving Feature Selection with Secure Multiparty Computation

Existing work on privacy-preserving machine learning with Secure Multipa...
07/01/2020

Private Speech Characterization with Secure Multiparty Computation

Deep learning in audio signal processing, such as human voice audio sign...
04/05/2020

FALCON: Honest-Majority Maliciously Secure Framework for Private Deep Learning

This paper aims to enable training and inference of neural networks in a...
04/08/2020

Improved Secure Efficient Delegated Private Set Intersection

Private Set Intersection (PSI) is a vital cryptographic technique used f...
07/14/2020

BUNET: Blind Medical Image Segmentation Based on Secure UNET

The strict security requirements placed on medical records by various pr...
05/03/2021

From Fairness to Full Security in Multiparty Computation

In the setting of secure multiparty computation (MPC), a set of mutually...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

In this work, we consider the problem of privacy-preserving outsourcing of machine learning computations where a set of clients

wish to jointly classify private inputs based on a machine learning model by outsourcing the computation to a set of cloud (compute) servers. We are interested in scenarios where the model (weights) can either be public (i.e., known to everyone) or private (i.e. provided as input by one of the clients).

More formally, we envision our protocol in three phases. In a first phase, the clients share their inputs with the cloud servers, followed by a compute phase, where the servers are tasked with applying a machine learning model jointly on the client’s inputs. In the third and final phase, the servers deliver the result of the inference back to the clients. Security here demands that classification remain private as long as there is one honest cloud server, however, correctness should hold even if all the servers are corrupted. In the context of outsourcing privacy-preserving machine learning computations past works have considered a weaker variant of this formulation where an honest server is required to guarantee correctness [MZ17, LJLA17, MRSV17]. With the exception of the work by Markri et al. [MRSV17], all previous works only consider passive security [MZ17, LJLA17, RWT18, JVC18, WGC18] where the adversary corrupting the cloud servers is assumed to honestly follow the protocol. The EPIC system from [MRSV17] achieves security in the face of active corruptions where the adversary is allowed to arbitrarily deviate. While EPIC does allow the data outsourcing party to pre-process its data via a neural network feature extractor, the network’s weights must be agnostic to the classification task. This constraint limits the neural network’s classification accuracy.

Providing security against active adversaries in our setting presents its own challenges beyond utilizing an active secure protocol for the underlying functionality. In more details, the parties must be ensured that the servers use the “right” inputs and do not abuse their valid inputs by adding to them a carefully chosen small adversarial perturbation with the aim to change the predication [TV16, KACK18]. These type of attacks can be devastating when correctness of computation is crucial to the application such as in medical diagnosis and image classification for defense applications, and only arise in the presence of active adversaries.

1.1 Background and Related Work

Deep neural networks, in general, implement multiple layers of computation. The inputs to a layer are multiplied by a matrix of trained weights, followed by an element-wise non-linear activation function. For classification tasks, the outputs of the final layer are passed through a special activation function, referred to as the

softmax

, that returns the probabilities of the input belonging to each class. Like prior work that applies cryptographic techniques on neural network computations 

[GBDL16, GGG17], we focus on neural networks that can be represented as arithmetic circuits

. Specifically, we constrain the inputs and weights of the network to be field elements and the activation functions to be quadratics, i.e., the output is the square of the input. We note that networks with quadratic activations have the same representational power as networks with more commonly used activations such as the ReLU function

[GNH16, LSSS14].

The study of privacy-preserving machine learning has recently witnessed prosperity due to the impressive progress in the design of practical secure multiparty computation (MPC) protocols, which mostly led to passively secure protocols thus far. We review some of these notable works. In [MZ17] Mohassel et al. introduce SecureML, a system for several privacy preserving machine learning training and classifications algorithms in the two-server model that run 2PC for arithmetic computation. In [LJLA17] Liu et al. develop MiniONN, a framework for transforming an existing neural network to an oblivious neural network that protects the privacy of the model (held by a cloud) and the client’s input in the predication phase. In [BSS17]

, somewhat homomorphic encryption is used for image classification focusing, on the particular non-linear Support Vector Machines algorithm. In

[RWT18] Riazi et al. present Chameleon, a system that supports hybrid secure computation in the two-party setting that combines arithmetic computation over rings for linear operations and Yao’s garbled circuits [Yao86]

for the non-linear computation. Chameleon provides training and classification for deep and convolutional neural networks. Juvekar et al.

[JVC18] extends this paradigm in GAZELLE for classifying private images using a convolutional neural network, protecting the classification phase and using fully homomorphic encryption scheme for carrying out the linear computation. This construction was later improved in [MRSV17] that introduced EPIC, an outsourcing scheme with active security. Finally, in a recent work by Wagh et al. [WGC18], the authors introduce SecureNN, a tool for training and predication in the three-party and four-party settings with honest majority. Their secure training implements three types neural networks, optimizing on prior training protocols.

Concretely efficient general secure two-party (2PC) and multi-party computation (MPC) has seen a lot of progress in the last decade with a focus on boolean computations [LP12, SS13, LPSY15, WRK17b, WRK17a]. However, in many applications, the computation can be more naturally described by using arithmetic operations over integers, real numbers, or other rings. For such instances of secure arithmetic computation, general techniques for securely evaluating Boolean circuits incur a very significant overhead (see, e.g., [AIK14, BMR16, Ben17] and references therein). For secure computation protocols that directly compute arithmetic functionalities, there have been two basic approaches: Oblivious-Transfer (OT) based [Gil99, KOS16, FPY18] and Oblivious Linear Evaluation111An OLE-based protocol employs a secure two-party protocol for computing the function over where is provided as input from one party and from the other. OLE is a commonly used primitive for secure arithmetic computation, analogously to the role of oblivious transfer in secure Boolean computation [NP99, IPS09, ADI17]. (OLE) based [DPSZ12, DGN17, GNN17, KPR18]. Generally speaking, the OT-based approach is light on the computation but involves a higher communication cost for secure arithmetic computation over large fields. On the other hand, the OLE-based implementations in prior works have required a specific OLE implementation and/or significant invocations of the OLE functionality per multiplication gate of the underlying arithmetic circuit. Moreover, all protocols have suffered a signficant overhead when moving from a passive secure protocol to an active one.

1.2 Our Contribution

In this work, we introduce a new model for securely outsourcing arithmetic computations to a set of untrusted cloud servers. We discuss an appealing use case of this application for outsourcing the classification of a (potentially) propriety neural network to two cloud servers. In our model, we provide secrecy when at least one cloud server is honest (i.e. non-colluding) and guarantee correctness even when all servers collude.

Next, we design a concretely efficient secure two-party computation protocol for general arithmetic circuits. A generic approach to achieve our goal would be for the servers to rely on a secure arithemtic computation protocol to compute the result and a zkSNARK proof of correctness of the computation [BCCT12]. However, this would be quite inefficient as the prover computations in these proof systems are typically heavy. Instead we rely on the secure arithmetic computation protocol of [HIMV18] where we show a zkSNARK proof can be attached with minimal overhead. In slight more detail, the work of [HIMV18] provide a secure two-party arithmetic computation protocol with significant efficiency improvements over previous protocols of this type. For example, for typical arithmetic circuits (that are large but not too narrow), it yields over 15x improvement over the recent Overdrive [KPR18] and TinyOLE [DGN17] protocols in terms of both communication and computation.

A distinctive feature of this protocol is that it can make a fully modular black-box use of any passively secure implementation of oblivious linear function evaluation (OLE), which serves as a natural building block for secure arithmetic computation. This means that it can build on a variety of existing or future implementations of passively secure OLE, inheriting their security and efficiency features. Given the multitude of optimization goals, security requirements and execution platforms, such a modular design can have major advantages.

We benchmark our protocol against our privacy-preserving machine learning system. As an instance of such a system, we train a two-layer arithmetic neural network that predicts the drug usage of patients based on data obtained from multiple health services providers including the patients’ personal and psychological traits and their usage of other drugs. The trained arithmetic neural network achieves accuracy and is comparable to that of a conventional neural network with floating point weights and ReLU activations.

2 Our Model

In our model, we consider different players who interact in synchronous rounds via authenticated point-to-point channels and a broadcast medium. Players can be designated three different roles: input clients who provide inputs, output clients who receive outputs, and servers who may be employed for the actual computation. Our protocols are described for the specific case of two servers but can be extended to constant number of servers following [IPS08]. We assume two different adversary models with different security guarantees:

  1. We consider a static rushing adversary that corrupts all but one of the input clients and all but one of the servers. In this scenario, we achieve security with abort, i.e. an adversary is restricted to modifying the inputs of corrupted input clients and can make the protocol abort but cannot violate the privacy of the inputs from uncorrupted input clients or correctness of the computation if it proceeds to completion.

  2. We consider a second scenario where the adversary is static and rushing and can corrupt all-but-one of the input clients and all the servers. In this scenario, the adversary can modify the inputs of corrupted input clients and learn the inputs of the uncorrupted clients but not affect the correctness of the output of the computation if it proceeds to completion.

2.1 Defining Security

In this section, we present the security definitions considered in this paper. We follow the standard simulation paradigm and define the real-world and ideal-world executions.

The real model.

Let be a multiparty protocol computing a deterministic functionality . Following [Can01], we consider the protocol execution in the presence of an adversary that is coordinated by an environment

that is modeled as non-uniform probabilistic polynomial time Turing machine. At the beginning of the computation, the environment chooses and delivers the inputs to the parties. More formally, the environment sends

to each party where and are respectively the computational and statistical security parameter. We denote by the set of corrupted parties. sends and to . The the parties interact according to protocol , where the honest parties follow the instructions as specified by which the corrupted parties behave arbitrarily as dictated by . At the end of the protocol provides an output to and additionally receives the output of the honest parties. Finally

outputs a bit and is denoted by the random variable

.

The ideal model - security with abort.

In the ideal model, we consider a trusted party that implements the functionality . Here too, we have an environment that provides the inputs to the honest parties and the inputs of corrupted parties and to an adversary . Execution in the ideal model proceeds as follows. Honest parties submit their input to the trusted party implementing . Corrupted parties can send arbitrary values as instructed by the adversary . We denote this by for . The trusted party computes and sends it to the adversary . The adversary then decides and instructs the functionality to deliver the output to the honest parties. As in the real world, the adversary sends an output to the environment. We denote the output of the environment ideal world by .

The ideal model - verifiable computation.

This ideal model is identical to the case of security with abort, with the only exception being, the adversary is given the inputs of all parties (not just the inputs of the corrupted parties). In other words, in this model, the adversary learns the inputs of all parties but cannot affect the correctness of the computation. We denote the output of the environment ideal world by .

Definition 1 (Security with abort).

Let be a protocol and be an -party functionality . We say that securely computes in the presence of static active adversaries corrupting the parties if for any PPT adversary there exists a PPT adversary such that for every non-uniform PPT there is a negligible function such that

Definition 2 (Verifiable Computation).

Let be a protocol and be an -party functionality . We say that securely computes in the presence of static active adversaries corrupting the parties if for any PPT adversary there exists a PPT adversary such that for every non-uniform PPT there is a negligible function such that

3 Our Secure Arithmetic Computation Protocol

In this work, we design, optimize, and implement an actively secure protocol for secure arithmetic computation based on the work of [HIMV18]. This protocol is based on the IPS-compiler, introduced by Ishai et al. [IPS08], which provides a general paradigm to design secure computation protocol combining the following two ingredients: (1) an active MPC protocol for the underlying function in the honest majority setting, and (2) a passive MPC protocol of a related functionality in the dishonest majority setting. The work of [HIMV18] optimizes both the analysis and the building blocks of the IPS-compiler and constructs a protocol which requires roughly 4 invocations of any passive OLE implementation per multiplication gate of the circuit. A crucial ingredient in the construction that enables the throughput is the use of Reed-Solomon codes to implement a packed secret sharing scheme [FY92].

First we recall the protocol from [HIMV18] that guarantees correctness and privacy when one of the parties remains honest. This is similar to the guarantees provided in the work of Makri et al. [MRSV17]. However, there is a subtle issue in their protocol that arises when outsourcing computations to malicious servers. To achieve security against all-but-one corruptions of the servers there needs to be a way to ensure that the inputs from all the parties have been correctly included in the computation (i.e. not been tampered with). Without such a mechanism, naively employing an actively secure protocol could allow an adversary to alter the input (via an “additive” attack) of any party before the computation starts. To mitigate such an attack, they employ a simple information theoretic mac. In slight more detail, they consider a slightly modified (randomized) functionality that the servers compute. Roughly speaking, the functionality takes as input MAC keys and and MACs and , in addition to the inputs and , and produces as output . The flag bit will be set to a random linear combination of the values and . Finally, we remark that this approach can be modularly combined with any 2PC, including the work of [MRSV17] to protect against such an attack.

In our protocol, we enhance this basic protocol from [HIMV18] to guarantee correctness even when all cloud servers are corrupt. In other words, in our protocol, an active adversary corrupting all cloud servers can potentially violate the privacy of the inputs but will be caught if it delivers an incorrect answer. In slight more detail, our approach takes advantage of the IPS compiler to additionally attach a “zero-knowledge” proof with the result of the computation. Roughly speaking, the views of a random subset of the parties emulated as part of the virtual outer protocol serves as the “proof”. This is inspired by the approach of Ishai et al. [IKOS07] and refined by Ames et al. [AHIV17]. Overall this variant requires two modifications to the underlying basic protocol: (1) Each party needs to publish a commitment (that can be implemented via a publicly verifiable “hash” function), and (2) The cloud servers compute a proof that attests that the inputs considered in the computation correspond to the public hash values released by the clients. In the full version of the paper, we provide the complete description of the protocol and security analysis. Formally, we provide security with abort if the adversary corrupts all-but-one of the servers according to Definition 1 and verifiable computation if all the servers are corrupted according to Definition 2. In this sense, we get a best of both worlds type of guarantee.

Finally, we re-iterate that most previous approaches for securely computing machine learning tasks have considered only passive security and the ones that consider active security cannot provide this additional verifiability feature in the presence of an active adversary corrupting all cloud servers.

Below, we benchmark the basic variant of our protocol to a concrete NN computation and leave implementing the enhanced version as future work. The details of the protocol from [HIMV18] is provided (verbatim) in Appendix A for immediate reference.

4 Implementation And Results

For evaluating our proposed model, we considered the Drug Consumption dataset [FMM17] obtained from the UCI repository. This dataset consists of drug usage records of drugs for respondents. For each respondent attributes are known, including personality traits obtained from psychological surveys, as well as personal information: level of education, age, gender, country of residence and ethnicity. We are envisioning a scenario in which personality traits are available to one health center, and drug usage for a number of drugs are available to another, and the two health centers want to collaboratively predict the risk of being a drug user for another drug. We trained a layer fully connected neural network with neurons in each layer to predict if a respondent is a user of LSD. We used of the data for training the the remaining for test. A quadratic function was used as the activation function in the hidden layers. Training the network on all features including personality traits, personal information as well as drug usage record for other drugs achieves a test accuracy of . It is worth noting that if each of the health centers have trained a model on the data available to them, the final accuracy would drop, with test accuracy when trained only on personality traits and personal information, and test accuracy when trained only on records for other drug usage and personal information. Using quadratic activations didn’t degrade the accuracy, as networks trained with ReLU activation achieved similar accuracies (, and respectively).

Benchmark.

We implemented our protocol in C++ using Shoup’s NTL library [Sho] to perform arithmetic over finite fields. We relied on a recent lightweight passive OLE implementation due to Catro et al. that is based on the LWE assumption [dCJV]. We implemented our protocol in C++ using Shoup’s NTL library [Sho]

to perform arithmetic over finite fields, in particular vector linear operations and discrete Fast Fourier Transforms. We chose an appropriate prime of length at least 60-bits with sufficiently many roots of unity. This is required for our packed secret sharing routine. Moreover, we constrained our parameters to fix the degree of the secret sharing in the outer protocol be a power of two. This is so that we achieve optimal efficiency through FFT algorithms in the finite field. We implemented symmetric encryption by xoring our plaintexts with a pseudorandom mask generated by NTL’s PRG implementation on input a 256 bit seed. In addition, we used the cryptoTools library

[Rin13] maintained by Peter Rindal for the commitment functionality, as well as network communication (which in turn is based on Boost Asio). We also used the libOTe library [OC13] for implementing the -out-of- oblivious transfer functionality which is maintained by Peter. We used the batched version of the (batched) passive OLE protocol due to Castro et al [dCJV] that is based on the LWE assumption for generating random OLE. If each server in the main protocol requires OLE invocations then with a -batched passive OLE protocol we invoke the passive OLE protocol times for each server for a total of invocations. We ran the server protocol between two Amazon EC2 machines, located in Ohio and N.Virigina. Both machines were Amazon EC2’s “r4.16xlarge” instance (2.3 GHz Intel Xeon E5 Broadwell Processors, 488Gb RAM) running Ubuntu 16.08. All our experiments were conducted in two phases. In the first phase, we ran the watchlist setup and generated random OLE as correlated outputs to the two parties and in the second phase we ran the main protocol to compute the desired functionality, consuming the random OLEs for every OLE invocation in the evaluation.

When the NN is assumed to be public, the entire computation (offline and online) took about 3.7s and total communication was 17.89MB. When the NN is private, it took  259s and the total communication was 3GB. We get added benefits when the computation is highly parallel and therefore we evaluated our system in the batched setting where we carry out the inference on several instances in parallel. The performance is reported in Figure 1. As the instances grows larger, we reach close 1s per instance and 4MB communication per instance when the model is public. We refer the reader to [HIMV18] for a detailed comparison of the basic secure arithmetic computation protocol with prior works.

Figure 1: Batched setting in Public NN.

References

  • [ADI17] Benny Applebaum, Ivan Damgård, Yuval Ishai, Michael Nielsen, and Lior Zichron. Secure arithmetic computation with constant computational overhead. In CRYPTO, pages 223–254, 2017.
  • [AHIV17] Scott Ames, Carmit Hazay, Yuval Ishai, and Muthuramakrishnan Venkitasubramaniam. Ligero: Lightweight sublinear arguments without a trusted setup. In CCS, pages 2087–2104, 2017.
  • [AIK14] Benny Applebaum, Yuval Ishai, and Eyal Kushilevitz. How to garble arithmetic circuits. SIAM J. Comput., 43(2):905–929, 2014.
  • [BCCT12] Nir Bitansky, Ran Canetti, Alessandro Chiesa, and Eran Tromer. From extractable collision resistance to succinct non-interactive arguments of knowledge, and back again. In Innovations in Theoretical Computer Science 2012, Cambridge, MA, USA, January 8-10, 2012, pages 326–349, 2012.
  • [Ben17] Aner Ben-Efraim. On multiparty garbling of arithmetic circuits. IACR Cryptology ePrint Archive, 2017:1186, 2017.
  • [BMR16] Marshall Ball, Tal Malkin, and Mike Rosulek. Garbling gadgets for boolean and arithmetic circuits. In CCS, pages 565–577, 2016.
  • [BSS17] Anthony Barnett, Jay Santokhi, Michael Simpson, Nigel P. Smart, Charlie Stainton-Bygrave, Srinivas Vivek, and Adrian Waller. Image classification using non-linear support vector machines on encrypted data. IACR Cryptology ePrint Archive, 2017:857, 2017.
  • [Can01] Ran Canetti. Universally composable security: A new paradigm for cryptographic protocols. In FOCS, pages 136–145, 2001.
  • [dCJV] Leo de Castro, Chiraag Juvekar, and Vinod Vaikuntanathan. Lightweight OLE implementation based on the LWE assumption.
  • [DGN17] Nico Döttling, Satrajit Ghosh, Jesper Buus Nielsen, Tobias Nilges, and Roberto Trifiletti. TinyOLE: Efficient actively secure two-party computation from oblivious linear function evaluation. In CCS, pages 2263–2276, 2017.
  • [DPSZ12] Ivan Damgård, Valerio Pastro, Nigel P. Smart, and Sarah Zakarias. Multiparty computation from somewhat homomorphic encryption. In CRYPTO, pages 643–662, 2012.
  • [FMM17] Elaine Fehrman, Awaz K Muhammad, Evgeny M Mirkes, Vincent Egan, and Alexander N Gorban. The five factor model of personality and evaluation of drug consumption risk. In Data Science, pages 231–242. Springer, 2017.
  • [FPY18] Tore Kasper Frederiksen, Benny Pinkas, and Avishay Yanai. Committed MPC - maliciously secure multiparty computation from homomorphic commitments. In PKC, pages 587–619, 2018.
  • [FY92] Matthew K. Franklin and Moti Yung. Communication complexity of secure computation (extended abstract). In STOC, pages 699–710, 1992.
  • [GBDL16] Ran Gilad-Bachrach, Nathan Dowlin, Kim Laine, Kristin Lauter, Michael Naehrig, and John Wernsing. Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy. In International Conference on Machine Learning, pages 201–210, 2016.
  • [GGG17] Zahra Ghodsi, Tianyu Gu, and Siddharth Garg. Safetynets: Verifiable execution of deep neural networks on an untrusted cloud. In NIPS, pages 4675–4684, 2017.
  • [Gil99] Niv Gilboa. Two party RSA key generation. In CRYPTO, pages 116–129, 1999.
  • [GMW87] Oded Goldreich, Silvio Micali, and Avi Wigderson. How to play any mental game or A completeness theorem for protocols with honest majority. In STOC, pages 218–229, 1987.
  • [GNH16] Antoine Gautier, Quynh N Nguyen, and Matthias Hein. Globally optimal training of generalized polynomial neural networks with nonlinear spectral methods. In Advances in Neural Information Processing Systems, pages 1687–1695, 2016.
  • [GNN17] Satrajit Ghosh, Jesper Buus Nielsen, and Tobias Nilges. Maliciously secure oblivious linear function evaluation with constant overhead. In ASIACRYPT, pages 629–659, 2017.
  • [HIMV18] Carmit Hazay, Yuval Ishai, Antonio Marcedone, and Muthuramakrishnan Venkitasubramaniam. Leviosa: Lightweight secure arithmetic computation, 2018.
  • [IKOS07] Yuval Ishai, Eyal Kushilevitz, Rafail Ostrovsky, and Amit Sahai. Zero-knowledge from secure multiparty computation. In STOC, pages 21–30, 2007.
  • [IPS08] Yuval Ishai, Manoj Prabhakaran, and Amit Sahai. Founding cryptography on oblivious transfer - efficiently. In CRYPTO, pages 572–591, 2008.
  • [IPS09] Yuval Ishai, Manoj Prabhakaran, and Amit Sahai. Secure arithmetic computation with no honest majority. In TCC, pages 294–314, 2009.
  • [JVC18] Chiraag Juvekar, Vinod Vaikuntanathan, and Anantha Chandrakasan. GAZELLE: A low latency framework for secure neural network inference. In USENIX, pages 1651–1669, 2018.
  • [KACK18] Felix Kreuk, Yossi Adi, Moustapha Cissé, and Joseph Keshet. Fooling end-to-end speaker verification with adversarial examples. In IEEE International Conference on Acoustics, Speech and Signal Processing, pages 1962–1966, 2018.
  • [KOS16] Marcel Keller, Emmanuela Orsini, and Peter Scholl. MASCOT: faster malicious arithmetic secure computation with oblivious transfer. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria, October 24-28, 2016, pages 830–842, 2016.
  • [KPR18] Marcel Keller, Valerio Pastro, and Dragos Rotaru. Overdrive: Making SPDZ great again. In EUROCRYPT, pages 158–189, 2018.
  • [LJLA17] Jian Liu, Mika Juuti, Yao Lu, and N. Asokan. Oblivious neural network predictions via minionn transformations. In CCS, pages 619–631, 2017.
  • [LP12] Yehuda Lindell and Benny Pinkas. Secure two-party computation via cut-and-choose oblivious transfer. J. Cryptology, 25(4):680–722, 2012.
  • [LPSY15] Yehuda Lindell, Benny Pinkas, Nigel P. Smart, and Avishay Yanai. Efficient constant round multi-party computation combining BMR and SPDZ. In CRYPTO, pages 319–338, 2015.
  • [LSSS14] Roi Livni, Shai Shalev-Shwartz, and Ohad Shamir. On the computational efficiency of training neural networks. In Advances in Neural Information Processing Systems, pages 855–863, 2014.
  • [MRSV17] Eleftheria Makri, Dragos Rotaru, Nigel P. Smart, and Frederik Vercauteren. Epic: Efficient private image classification (or: Learning from the masters). IACR Cryptology ePrint Archive, 2017:1190, 2017.
  • [MZ17] Payman Mohassel and Yupeng Zhang. Secureml: A system for scalable privacy-preserving machine learning. In IEEE Symposium on Security and Privacy, pages 19–38, 2017.
  • [NP99] Moni Naor and Benny Pinkas. Oblivious transfer and polynomial evaluation. In STOC, pages 245–254, 1999.
  • [OC13] OSU-Crypto. libOTe. https://github.com/osu-crypto/libOTe, 2013.
  • [Rin13] P. Rindal. Cryptotools. https://github.com/ladnir/cryptoTools, 2013.
  • [RWT18] M. Sadegh Riazi, Christian Weinert, Oleksandr Tkachenko, Ebrahim M. Songhori, Thomas Schneider, and Farinaz Koushanfar. Chameleon: A hybrid secure computation framework for machine learning applications. In AsiaCCS, pages 707–721, 2018.
  • [Sho] Victor Shoup. NTL library. http://www.shoup.net/ntl/.
  • [SS13] Abhi Shelat and Chih-Hao Shen. Fast two-party secure computation with minimal assumptions. In CCS, pages 523–534, 2013.
  • [TV16] Pedro Tabacof and Eduardo Valle. Exploring the space of adversarial images. In International Joint Conference on Neural Networks, pages 426–433, 2016.
  • [WGC18] Sameer Wagh, Divya Gupta, and Nishanth Chandran. Securenn: Efficient and private neural network training. IACR Cryptology ePrint Archive, 2018:442, 2018.
  • [WRK17a] Xiao Wang, Samuel Ranellucci, and Jonathan Katz. Authenticated garbling and efficient maliciously secure multi-party computation. In Proc. ACM CCS, to appear, 2017. Full version: Cryptology ePrint Archive, Report 2017/030.
  • [WRK17b] Xiao Wang, Samuel Ranellucci, and Jonathan Katz. Authenticated garbling and efficient maliciously secure two-party computation. In CCS, pages 21–37, 2017.
  • [Yao86] Andrew Chi-Chih Yao. How to generate and exchange secrets (extended abstract). In FOCS, pages 162–167, 1986.

Appendix A Secure Arithmetic Computation Protocol of [Himv18]

In this section, we provide the secure arithmetic computation protocol of [HIMV18] (verbatim) for immediate reference. The basic protocol is obtained by modularly combining two ingredients: (1) an outer protocol that is secure in the honest majority setting against active adversaries, and (2) an inner protocol that is secure in the dishonest majority setting against passive adversaries. We describe the outer and inner protocol first.

a.1 Our Optimized Outer Protocol

Inputs. Client ’s input is and client ’s input is . The parties share a description of an arithmetic circuit that implements .

Notations. denote the block length, denote the privacy parameter of the protocol and is the robustness parameter. We set is the degree of the polynomials used to shares the blocks and the number servers is .

The protocol. The protocol proceeds iteratively starting from the input layer to the last layer

Input sharing. The clients and arrange their inputs values into blocks of length with a replication pattern that matches the parallel evaluation of the first layer. Namely, for each input block of (resp. of ), it samples a random codeword such that , by sampling a random polynomial of degree such that . It then sends to server .

Evaluating the layer of . Iterating through the layers, the servers do the following for layers . The input wires for the layer are organized left and right blocks corresponding to left and right input wires, where the output of each layer can be obtained by performing an arithmetic operation on pairs of input blocks. Furthermore, we assume that each block corresponds to a vector such that is held by server and allegedly encodes the values for the wires within the block. Formally,

Addition/Subtraction. Addition or subtraction of the corresponding left and right blocks, respectively denoted by and , are handled without any interaction between the servers, by simply having each server locally add or subtract the corresponding shares. Namely, where and are the vector of shares corresponding to and .

Multiplication. Multiplication of blocks and is more involved and requires interaction between the clients. Let and be the vector of shares corresponding to these blocks. First, the servers locally multiply their shares and to obtain . If and belong to , then . The servers then perform a degree reduction to obtain a fresh code word for the same block that lies in .

Degree reduction.

Since degree reduction can be expressed as a linear transformation on the vector

. Namely, there exists a public matrix such that where if , then and . More precisely, to perform a degree reduction, server computes a 2-out-of-2 additive sharing of , namely and sends to and to . collects and computes . Next, the client samples a random codeword in that encodes the all 0s vector and sends to where . performs an analogous computation and sends to the servers. The servers locally add the shares received from the clients.

Rearranging the blocks for next layer. Upon completing the computation of some layer, the output blocks are rearranged for the computation in the next layer. This is done the same way as in the above degree reduction, as permuting the elements of a vector that encodes a block can be achieved by a linear transformation.

Correctness tests. See Figure 3.

Output sharing. Each server sends the clients its shares corresponding to the output blocks. The clients reconstruct the secrets and obtain the final outputs.

Figure 2: Optimized Outer Protocol

After computing all layers, the servers perform the following tests: degree test, permutation test and equality test. Each of these tests are performed times where is an input parameter.

Notations. A codeword encodes a block of secrets that sum up to 0 if . A codeword encodes the all 0s block if for all .

Degree test. This test verifies that the vectors corresponding to all input and output blocks of each layer are valid codewords, namely belong to . (We remark that we do not consider the output blocks of the multiplication layers (as they are in ), rather only the output blocks upon performing the degree reduction.) The clients first distribute the vectors and that allegedly encode random blocks. That is, each server receives the values and from the respective party and . Let denote the matrix that contains the two rows , and the tested blocks . The servers then receive random field elements from the coin-tossing oracle and locally compute . That is, each server , holding a column , locally computes and broadcasts to all other servers. The servers collect the vector and abort if . Otherwise they proceed to the permutation test.

Permutation test. This test verifies that the input blocks of each layer correctly encode the values from the output blocks of the previous layer. Once again, we will consider the matrix as in the degree test with the exception that the vectors and encode random secrets that sum up to within each block. Observe that the set of all permutation constraints can be abstracted as a set of public linear constraints and a vector such that where is the concatenation of the vectors corresponding to the inputs and outputs of each layer throughout the computation that are encoded within .

To check this constraint, each server receives a random vector from the coin-tossing oracle and computes

Now, let be the unique polynomial of degree such that for every and . Then server locally computes and broadcasts it to the other servers. The servers collect the values and abort if or where .

Equality test. In the equality test, the servers check that the degree reduction was performed correctly. This procedure will be similar to the permutation test but simpler. Namely, each server defines two matrices and where contains the vectors in and contains the vectors after the degree reduction, namely in . The servers receive from the coin-tossing oracle and compute the polynomials that encode as above. Next, server computes

and broadcasts . The servers then collect the values and aborts if does not encode the all 0s block.

Figure 3: Correctness Tests for Protocol

In this section we present our optimized outer protocol in the honest majority setting which involves two clients and and servers. We consider a slight variant of the IPS compiler where we allow the servers in the outer protocol to have access to a coin-tossing oracle which upon invocation can broadcast random values to all servers. When compiling this variant, this oracle is implemented via a coin-tossing protocol executed between the clients (cf. Figure 4). A crucial ingredient in our construction is the use of Reed-Solomon codes for computing packed secret shares [FY92]. We start by providing our coding notations and related definitions.

Coding notation.

For a code and vector , denote by the minimal distance of from , namely the number of positions in which differs from the closest codeword in , and by the set of positions in which differs from such a closest codeword (in case of a tie, take the lexicographically first closest codeword). We further denote by the minimal distance between a vector set and a code , namely .

Definition 3 (Reed-Solomon code).

For positive integers , finite field , and a vector of distinct field elements, the code is the linear code over that consists of all -tuples where is a polynomial of degree over .

Definition 4 (Encoded message).

Let be an RS code and be a sequence of distinct elements of for . For we define the message to be , where is the polynomial (of degree ) corresponding to . For with rows , we let be the length vector such that for . We say that encodes if .

We further denote by replication pattern the replication induced by structure of circuit between every pair of layers. A formal description of our protocol is given in Figures 2, 3.

We now have the following theorem for our protocol.

Theorem 1.

Let and let be a two-party functionality, then protocol from Figure 2 securely computes tolerating static active corruption of at most one client and adaptive active corruptions of at most servers and passive corruptions of at most servers with statistical security of (where is a soundness amplification parameter).

a.2 The Inner Protocol

Recall that in the IPS compiler, the inner protocol is a two-party protocol executed between the two parties and and security is required to hold only against corruption by a passive adversary. Furthermore, the functionalities considered are precisely the next message functions executed by the servers in the outer protocol. On a high-level, the state of each of server is maintained jointly by the parties where each holds a share. Emulating the internal computation of each server for our outer protocol boils down to securely updating the states of the servers based on the computation specified in Figures 2 and 3. We remark that all computations performed by the servers are arithmetic computations over the same field. For the inner protocol and we will rely on the GMW protocol [GMW87] described in the (passive) OLE-hybrid, where the OLE functionality can be instantiated with any passively secure protocol [NP99, IPS09].

a.3 The Combined Protocol

In this section we provide our complete two party protocol for realizing arithmetic functions over any field that achieve security against active corruptions. This is obtained by compiling our outer protocol described in A.1 and the inner protocol instantiated using the GMW protocol [GMW87] with a variant of the IPS compiler. We recall that, in IPS compiler, the parties executing the protocol will play the respecitve roles in the inner protocol, and play the roles of the clients in the outer protocol. The actual server actions are never executed by a single party. Instead, the state of the servers are shared using an additive secret sharing scheme between the parties and each server computation is emulated jointly by the two parties via the inner protocol to update the state. Finally, an important part of the compiler is the watchist mechanism where each party monitors the actions of the other party when emulating the servers. Roughly speaking, each party knows the inputs and randomness used by the other party to emulate the server for some hidden subset of the servers refered to as as the watchlist. A party is caught if it deviates in executing the inner protocol corresponding to a server that is on the other party’s watchlist.

A formal description of the protocol is given in Figure 4.

The Combined Protocol Inputs. ’s input is and ’s input is . The parties share a description of an arithmetic circuit that implements . Watchlists setup. To establish the watchlist, and run two instances of an actively secure -out-of- oblivious-transfer (OT) protocol where is the privacy parameter of the outer protocol. In one instance plays the role of the sender with symmetric keys as input and plays the receiver with an arbitrary -subset of as its watchlist. In the second instance, the parties execute the same protocol with the roles reversed. The symmetric keys will be used to encrypt the input and randomness used by the party for executing the inner protocol corresponding to the servers where the information for the server is encrypted with the key. This way each party learns the information corresponding to the servers on their watchlists and monitors the behaviour of the other party. Input sharing. For each , secret shares its input using an additively secret sharing scheme. Namely, picks pairs of random field elements subject to for all , and sends to . Similarly, secret shares its input and sends the shares to . Protocol emulation. The parties proceed iteratively starting from the input layer to the last layer , emulating the actions of the clients and servers in Protocol . In more details, the parties play the roles of the clients exactly as carried out in . To emulate the computations of the servers, the parties maintain throughput the emulation of additive shares of the local state of each server, that contains the incoming and outgoing messages until that point. Then, additions are emulated locally by adding the corresponding shares. Next, to emulate the multiplications carried out by each server, the parties employ the inner protocol to multiply from input shares of the elements to receive shares of the product. This is the GMW protocol that makes use of two OLE calls. Finally, to emulate the communication from to , encrypts the message to be sent from to using secret key and sends the ciphertext to the other party. To simulate a server communicates a message to a client, the other client simply reveals it share of the message being communicated. Coin tossing. Whenever the outer protocol invokes the coin-tossing oracle , the parties run a coin tossing protocol using the commitment functionality , for producing the randomness that is used as the randomness obtained from the coin tossing oracle for the outer protocol. Namely, each party commits to each share, upon both committing the parties open their commitments and XOR the shares. Checking consistency. Each party who has server in its watchlist carries out a consistency check between the messages reported over the watchlist channel and the messages it receives from . Specifically, party recomputes the messages sent by using the messages reported on the watchlist channel , and aborts in case of detecting inconsistency.

Figure 4: Arithmetic 2PC with active security

We conclude with the following theorem.

Theorem 2.

Let be a two-party functionality with depth . Let be the outer MPC protocol described in Section A.1 for realizing with servers, -robustness, -privacy, repetitions, block length and . Let be the multiplication functionality that takes additive shares as inputs from the parties and outputs additive shares of the product and a two-party protocol that realizes in the passive OLE-hybrid setting with perfect security against a passive corruption of either party. Then, if and , the compiled protocol from Figure 4, denoted by , securely realizes in the passive OLE-hybrid model against active (static) corruption of either party with statistical security

The communication complexity of is

where is the communication complexity of and is a soundness amplification parameter. In addition, the number of OLE invocations of is where is the number of OLE invocations of .