Breaking the Communication-Privacy-Accuracy Trilemma

by   Wei-Ning Chen, et al.

Two major challenges in distributed learning and estimation are 1) preserving the privacy of the local samples; and 2) communicating them efficiently to a central server, while achieving high accuracy for the end-to-end task. While there has been significant interest in addressing each of these challenges separately in the recent literature, treatments that simultaneously address both challenges are still largely missing. In this paper, we develop novel encoding and decoding mechanisms that simultaneously achieve optimal privacy and communication efficiency in various canonical settings. In particular, we consider the problems of mean estimation and frequency estimation under ε-local differential privacy and b-bit communication constraints. For mean estimation, we propose a scheme based on Kashin's representation and random sampling, with order-optimal estimation error under both constraints. For frequency estimation, we present a mechanism that leverages the recursive structure of Walsh-Hadamard matrices and achieves order-optimal estimation error for all privacy levels and communication budgets. As a by-product, we also construct a distribution estimation mechanism that is rate-optimal for all privacy regimes and communication constraints, extending recent work that is limited to b=1 and ε=O(1). Our results demonstrate that intelligent encoding under joint privacy and communication constraints can yield a performance that matches the optimal accuracy achievable under either constraint alone.


page 1

page 2

page 3

page 4


Communication Efficient, Sample Optimal, Linear Time Locally Private Discrete Distribution Estimation

We consider discrete distribution estimation over k elements under ε-loc...

A normal approximation for joint frequency estimatation under Local Differential Privacy

In the recent years, Local Differential Privacy (LDP) has been one of th...

Exact Optimality of Communication-Privacy-Utility Tradeoffs in Distributed Mean Estimation

We study the mean estimation problem under communication and local diffe...

Privacy Amplification via Compression: Achieving the Optimal Privacy-Accuracy-Communication Trade-off in Distributed Mean Estimation

Privacy and communication constraints are two major bottlenecks in feder...

Discrete Distribution Estimation under User-level Local Differential Privacy

We study discrete distribution estimation under user-level local differe...

Privacy for Free: Communication-Efficient Learning with Differential Privacy Using Sketches

Communication and privacy are two critical concerns in distributed learn...

Trends and Advancements in Deep Neural Network Communication

Due to their great performance and scalability properties neural network...

Please sign up or login with your details

Forgot password? Click here to reset