Shuffle Private Stochastic Convex Optimization

by   Albert Cheu, et al.

In shuffle privacy, each user sends a collection of randomized messages to a trusted shuffler, the shuffler randomly permutes these messages, and the resulting shuffled collection of messages must satisfy differential privacy. Prior work in this model has largely focused on protocols that use a single round of communication to compute algorithmic primitives like means, histograms, and counts. In this work, we present interactive shuffle protocols for stochastic convex optimization. Our optimization protocols rely on a new noninteractive protocol for summing vectors of bounded ℓ_2 norm. By combining this sum subroutine with techniques including mini-batch stochastic gradient descent, accelerated gradient descent, and Nesterov's smoothing method, we obtain loss guarantees for a variety of convex loss functions that significantly improve on those of the local model and sometimes match those of the central model.


page 1

page 2

page 3

page 4


Improved Rates for Differentially Private Stochastic Convex Optimization with Heavy-Tailed Data

We study stochastic convex optimization with heavy-tailed data under the...

Pure Differential Privacy from Secure Intermediaries

Recent work in differential privacy has explored the prospect of combini...

Jensen: An Easily-Extensible C++ Toolkit for Production-Level Machine Learning and Convex Optimization

This paper introduces Jensen, an easily extensible and scalable toolkit ...

Stability Enhanced Privacy and Applications in Private Stochastic Gradient Descent

Private machine learning involves addition of noise while training, resu...

Private Stochastic Convex Optimization: Efficient Algorithms for Non-smooth Objectives

In this paper, we revisit the problem of private stochastic convex optim...

Privacy of Noisy Stochastic Gradient Descent: More Iterations without More Privacy Loss

A central issue in machine learning is how to train models on sensitive ...

Improving upon NBA point-differential rankings

For some time, point-differential has been thought to be a better predic...