An Adaptive and Fast Convergent Approach to Differentially Private Deep Learning

12/19/2019
by   Zhiying Xu, et al.
0

With the advent of the era of big data, deep learning has become a prevalent building block in a variety of machine learning or data mining tasks, such as signal processing, network modeling and traffic analysis, to name a few. The massive user data crowdsourced plays a crucial role in the success of deep learning models. However, it has been shown that user data may be inferred from trained neural models and thereby exposed to potential adversaries, which raises information security and privacy concerns. To address this issue, recent studies leverage the technique of differential privacy to design private-preserving deep learning algorithms. Albeit successful at privacy protection, differential privacy degrades the performance of neural models. In this paper, we develop ADADP, an adaptive and fast convergent learning algorithm with a provable privacy guarantee. ADADP significantly reduces the privacy cost by improving the convergence speed with an adaptive learning rate and mitigates the negative effect of differential privacy upon the model accuracy by introducing adaptive noise. The performance of ADADP is evaluated on real-world datasets. Experiment results show that it outperforms state-of-the-art differentially private approaches in terms of both privacy cost and model accuracy.

READ FULL TEXT
research
03/31/2020

Differentially Private Naïve Bayes Classifier using Smooth Sensitivity

With the increasing collection of users' data, protecting individual pri...
research
03/10/2021

Quantum machine learning with differential privacy

Quantum machine learning (QML) can complement the growing trend of using...
research
07/14/2021

Towards Quantifying the Carbon Emissions of Differentially Private Machine Learning

In recent years, machine learning techniques utilizing large-scale datas...
research
05/09/2019

Differentially Private Learning with Adaptive Clipping

We introduce a new adaptive clipping technique for training learning mod...
research
09/07/2022

Privacy-Preserving Deep Learning Model for Covid-19 Disease Detection

Recent studies demonstrated that X-ray radiography showed higher accurac...
research
07/12/2021

Improving the Algorithm of Deep Learning with Differential Privacy

In this paper, an adjustment to the original differentially private stoc...
research
05/08/2023

Differentially Private Attention Computation

Large language models (LLMs) have had a profound impact on numerous aspe...

Please sign up or login with your details

Forgot password? Click here to reset