Improving the Algorithm of Deep Learning with Differential Privacy

07/12/2021
by   Mehdi Amian, et al.
0

In this paper, an adjustment to the original differentially private stochastic gradient descent (DPSGD) algorithm for deep learning models is proposed. As a matter of motivation, to date, almost no state-of-the-art machine learning algorithm hires the existing privacy protecting components due to otherwise serious compromise in their utility despite the vital necessity. The idea in this study is natural and interpretable, contributing to improve the utility with respect to the state-of-the-art. Another property of the proposed technique is its simplicity which makes it again more natural and also more appropriate for real world and specially commercial applications. The intuition is to trim and balance out wild individual discrepancies for privacy reasons, and at the same time, to preserve relative individual differences for seeking performance. The idea proposed here can also be applied to the recurrent neural networks (RNN) to solve the gradient exploding problem. The algorithm is applied to benchmark datasets MNIST and CIFAR-10 for a classification task and the utility measure is calculated. The results outperformed the original work.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/08/2020

Removing Disparate Impact of Differentially Private Stochastic Gradient Descent on Model Accuracy

When we enforce differential privacy in machine learning, the utility-pr...
research
12/25/2017

On Connecting Stochastic Gradient MCMC and Differential Privacy

Significant success has been realized recently on applying machine learn...
research
12/19/2019

An Adaptive and Fast Convergent Approach to Differentially Private Deep Learning

With the advent of the era of big data, deep learning has become a preva...
research
03/03/2023

Exploring Machine Learning Privacy/Utility trade-off from a hyperparameters Lens

Machine Learning (ML) architectures have been applied to several applica...
research
02/22/2018

The Secret Sharer: Measuring Unintended Neural Network Memorization & Extracting Secrets

Machine learning models based on neural networks and deep learning are b...
research
07/09/2019

Characterizing Inter-Layer Functional Mappings of Deep Learning Models

Deep learning architectures have demonstrated state-of-the-art performan...

Please sign up or login with your details

Forgot password? Click here to reset