Adam in Private: Secure and Fast Training of Deep Neural Networks with Adaptive Moment Estimation

06/04/2021
by   Nuttapong Attrapadung, et al.
0

Privacy-preserving machine learning (PPML) aims at enabling machine learning (ML) algorithms to be used on sensitive data. We contribute to this line of research by proposing a framework that allows efficient and secure evaluation of full-fledged state-of-the-art ML algorithms via secure multi-party computation (MPC). This is in contrast to most prior works, which substitute ML algorithms with approximated "MPC-friendly" variants. A drawback of the latter approach is that fine-tuning of the combined ML and MPC algorithms is required, which might lead to less efficient algorithms or inferior quality ML. This is an issue for secure deep neural networks (DNN) training in particular, as this involves arithmetic algorithms thought to be "MPC-unfriendly", namely, integer division, exponentiation, inversion, and square root. In this work, we propose secure and efficient protocols for the above seemingly MPC-unfriendly computations. Our protocols are three-party protocols in the honest-majority setting, and we propose both passively secure and actively secure with abort variants. A notable feature of our protocols is that they simultaneously provide high accuracy and efficiency. This framework enables us to efficiently and securely compute modern ML algorithms such as Adam and the softmax function "as is", without resorting to approximations. As a result, we obtain secure DNN training that outperforms state-of-the-art three-party systems; our full training is up to 6.7 times faster than just the online phase of the recently proposed FALCON@PETS'21 on a standard benchmark network. We further perform measurements on real-world DNNs, AlexNet and VGG16. The performance of our framework is up to a factor of about 12-14 faster for AlexNet and 46-48 faster for VGG16 to achieve an accuracy of 70 FALCON.

READ FULL TEXT
research
01/02/2019

Secure Computation for Machine Learning With SPDZ

Secure Multi-Party Computation (MPC) is an area of cryptography that ena...
research
02/11/2020

Privacy-preserving collaborative machine learning on genomic data using TensorFlow

Machine learning (ML) methods have been widely used in genomic studies. ...
research
04/02/2021

PolyDNN: Polynomial Representation of NN for Communication-less SMPC Inference

The structure and weights of Deep Neural Networks (DNN) typically encode...
research
07/08/2019

QUOTIENT: Two-Party Secure Neural Network Training and Prediction

Recently, there has been a wealth of effort devoted to the design of sec...
research
05/20/2022

SafeNet: Mitigating Data Poisoning Attacks on Private Machine Learning

Secure multiparty computation (MPC) has been proposed to allow multiple ...
research
09/27/2022

MPC-Pipe: an Efficient Pipeline Scheme for Secure Multi-party Machine Learning Inference

Multi-party computing (MPC) has been gaining popularity over the past ye...
research
07/24/2020

MPC-enabled Privacy-Preserving Neural Network Training against Malicious Attack

In the past decades, the application of secure multiparty computation (M...

Please sign up or login with your details

Forgot password? Click here to reset