AESPA: Accuracy Preserving Low-degree Polynomial Activation for Fast Private Inference

01/18/2022
by   Jaiyoung Park, et al.
0

Hybrid private inference (PI) protocol, which synergistically utilizes both multi-party computation (MPC) and homomorphic encryption, is one of the most prominent techniques for PI. However, even the state-of-the-art PI protocols are bottlenecked by the non-linear layers, especially the activation functions. Although a standard non-linear activation function can generate higher model accuracy, it must be processed via a costly garbled-circuit MPC primitive. A polynomial activation can be processed via Beaver's multiplication triples MPC primitive but has been incurring severe accuracy drops so far. In this paper, we propose an accuracy preserving low-degree polynomial activation function (AESPA) that exploits the Hermite expansion of the ReLU and basis-wise normalization. We apply AESPA to popular ML models, such as VGGNet, ResNet, and pre-activation ResNet, to show an inference accuracy comparable to those of the standard models with ReLU activation, achieving superior accuracy over prior low-degree polynomial studies. When applied to the all-RELU baseline on the state-of-the-art Delphi PI protocol, AESPA shows up to 42.1x and 28.3x lower online latency and communication cost.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/14/2023

Fast and Private Inference of Deep Neural Networks by Co-designing Activation Functions

Machine Learning as a Service (MLaaS) is an increasingly popular design ...
research
11/11/2020

On Polynomial Approximations for Privacy-Preserving and Verifiable ReLU Networks

Outsourcing neural network inference tasks to an untrusted cloud raises ...
research
09/09/2023

Compact: Approximating Complex Activation Functions for Secure Computation

Secure multi-party computation (MPC) techniques can be used to provide d...
research
09/20/2022

PolyMPCNet: Towards ReLU-free Neural Architecture Search in Two-party Computation Based Private Inference

The rapid growth and deployment of deep learning (DL) has witnessed emer...
research
10/22/2020

CryptoGRU: Low Latency Privacy-Preserving Text Analysis With GRU

Billions of text analysis requests containing private emails, personal t...
research
10/18/2022

NFGen: Automatic Non-linear Function Evaluation Code Generator for General-purpose MPC Platforms

Due to the absence of a library for non-linear function evaluation, so-c...
research
02/13/2022

Highly Scalable Beaver Triple Generator from Additive-only Homomorphic Encryption

In a convolution neural network, a composition of linear scalar product,...

Please sign up or login with your details

Forgot password? Click here to reset