Differentially Private ADMM Algorithms for Machine Learning

10/31/2020
by   Tao Xu, et al.
18

In this paper, we study efficient differentially private alternating direction methods of multipliers (ADMM) via gradient perturbation for many machine learning problems. For smooth convex loss functions with (non)-smooth regularization, we propose the first differentially private ADMM (DP-ADMM) algorithm with performance guarantee of (ϵ,δ)-differential privacy ((ϵ,δ)-DP). From the viewpoint of theoretical analysis, we use the Gaussian mechanism and the conversion relationship between Rényi Differential Privacy (RDP) and DP to perform a comprehensive privacy analysis for our algorithm. Then we establish a new criterion to prove the convergence of the proposed algorithms including DP-ADMM. We also give the utility analysis of our DP-ADMM. Moreover, we propose an accelerated DP-ADMM (DP-AccADMM) with the Nesterov's acceleration technique. Finally, we conduct numerical experiments on many real-world datasets to show the privacy-utility tradeoff of the two proposed algorithms, and all the comparative analysis shows that DP-AccADMM converges faster and has a better utility than DP-ADMM, when the privacy budget ϵ is larger than a threshold.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

08/11/2020

Towards Plausible Differentially Private ADMM Based Distributed Machine Learning

The Alternating Direction Method of Multipliers (ADMM) and its distribut...
08/30/2018

DP-ADMM: ADMM-based Distributed Learning with Differential Privacy

Distributed machine learning is making great changes in a wide variety o...
11/26/2019

Gradient Perturbation is Underrated for Differentially Private Convex Optimization

Gradient perturbation, widely used for differentially private optimizati...
02/03/2020

Differentially Private k-Means Clustering with Guaranteed Convergence

Iterative clustering algorithms help us to learn the insights behind the...
12/24/2021

DP-UTIL: Comprehensive Utility Analysis of Differential Privacy in Machine Learning

Differential Privacy (DP) has emerged as a rigorous formalism to reason ...
09/21/2021

Privacy, Security, and Utility Analysis of Differentially Private CPES Data

Differential privacy (DP) has been widely used to protect the privacy of...
09/18/2019

Renyi Differentially Private ADMM for Non-Smooth Regularized Optimization

In this paper we consider the problem of minimizing composite objective ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.