Adaptive Differentially Private Empirical Risk Minimization

10/14/2021
by   Xiaoxia Wu, et al.
12

We propose an adaptive (stochastic) gradient perturbation method for differentially private empirical risk minimization. At each iteration, the random noise added to the gradient is optimally adapted to the stepsize; we name this process adaptive differentially private (ADP) learning. Given the same privacy budget, we prove that the ADP method considerably improves the utility guarantee compared to the standard differentially private method in which vanilla random noise is added. Our method is particularly useful for gradient-based algorithms with time-varying learning rates, including variants of AdaGrad (Duchi et al., 2011). We provide extensive numerical experiments to demonstrate the effectiveness of the proposed adaptive differentially private algorithm.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 31

10/20/2017

Differentially Private Empirical Risk Minimization with Input Perturbation

We propose a novel framework for the differentially private ERM, input p...
08/20/2019

AdaCliP: Adaptive Clipping for Private SGD

Privacy preserving machine learning algorithms are crucial for learning ...
08/05/2020

Differentially Private Accelerated Optimization Algorithms

We present two classes of differentially private optimization algorithms...
04/23/2021

On a Utilitarian Approach to Privacy Preserving Text Generation

Differentially-private mechanisms for text generation typically add care...
06/25/2021

Private Adaptive Gradient Methods for Convex Optimization

We study adaptive methods for differentially private convex optimization...
03/02/2021

DPlis: Boosting Utility of Differentially Private Deep Learning via Randomized Smoothing

Deep learning techniques have achieved remarkable performance in wide-ra...
03/29/2017

Efficient Private ERM for Smooth Objectives

In this paper, we consider efficient differentially private empirical ri...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.