DeepAI AI Chat
Log In Sign Up

Adaptive Differentially Private Empirical Risk Minimization

by   Xiaoxia Wu, et al.

We propose an adaptive (stochastic) gradient perturbation method for differentially private empirical risk minimization. At each iteration, the random noise added to the gradient is optimally adapted to the stepsize; we name this process adaptive differentially private (ADP) learning. Given the same privacy budget, we prove that the ADP method considerably improves the utility guarantee compared to the standard differentially private method in which vanilla random noise is added. Our method is particularly useful for gradient-based algorithms with time-varying learning rates, including variants of AdaGrad (Duchi et al., 2011). We provide extensive numerical experiments to demonstrate the effectiveness of the proposed adaptive differentially private algorithm.


A Simple Differentially Private Algorithm for Global Minimum Cut

In this note, we present a simple differentially private algorithm for t...

Differentially Private Optimization for Smooth Nonconvex ERM

We develop simple differentially private optimization algorithms that mo...

Adaptive False Discovery Rate Control with Privacy Guarantee

Differentially private multiple testing procedures can protect the infor...

Differentially Private Linear Sketches: Efficient Implementations and Applications

Linear sketches have been widely adopted to process fast data streams, a...

Differentially Private Accelerated Optimization Algorithms

We present two classes of differentially private optimization algorithms...

On a Utilitarian Approach to Privacy Preserving Text Generation

Differentially-private mechanisms for text generation typically add care...

Private Adaptive Gradient Methods for Convex Optimization

We study adaptive methods for differentially private convex optimization...