How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy

03/01/2023
by   Natalia Ponomareva, et al.
0

ML models are ubiquitous in real world applications and are a constant focus of research. At the same time, the community has started to realize the importance of protecting the privacy of ML training data. Differential Privacy (DP) has become a gold standard for making formal statements about data anonymization. However, while some adoption of DP has happened in industry, attempts to apply DP to real world complex ML models are still few and far between. The adoption of DP is hindered by limited practical guidance of what DP protection entails, what privacy guarantees to aim for, and the difficulty of achieving good privacy-utility-computation trade-offs for ML models. Tricks for tuning and maximizing performance are scattered among papers or stored in the heads of practitioners. Furthermore, the literature seems to present conflicting evidence on how and whether to apply architectural adjustments and which components are "safe" to use with DP. This work is a self-contained guide that gives an in-depth overview of the field of DP ML and presents information about achieving the best possible DP ML model with rigorous privacy guarantees. Our target audience is both researchers and practitioners. Researchers interested in DP for ML will benefit from a clear overview of current advances and areas for improvement. We include theory-focused sections that highlight important topics such as privacy accounting and its assumptions, and convergence. For a practitioner, we provide a background in DP theory and a clear step-by-step guide for choosing an appropriate privacy definition and approach, implementing DP training, potentially updating the model architecture, and tuning hyperparameters. For both researchers and practitioners, consistently and fully reporting privacy guarantees is critical, and so we propose a set of specific best practices for stating guarantees.

READ FULL TEXT

page 33

page 36

page 39

page 40

research
06/09/2022

A Critical Review on the Use (and Misuse) of Differential Privacy in Machine Learning

We review the use of differential privacy (DP) for privacy protection in...
research
05/19/2022

Differential Privacy: What is all the noise about?

Differential Privacy (DP) is a formal definition of privacy that provide...
research
02/21/2022

Personalized PATE: Differential Privacy for Machine Learning with Individual Privacy Guarantees

Applying machine learning (ML) to sensitive domains requires privacy pro...
research
11/27/2020

An Integrated Approach Towards the Construction of an HCI Methodological Framework

We present a methodological framework aiming at the support of HCI pract...
research
07/26/2022

Lifelong DP: Consistently Bounded Differential Privacy in Lifelong Machine Learning

In this paper, we show that the process of continually learning new task...
research
11/21/2022

Privacy in Practice: Private COVID-19 Detection in X-Ray Images

Machine learning (ML) can help fight the COVID-19 pandemic by enabling r...
research
11/28/2022

The Grind for Good Data: Understanding ML Practitioners' Struggles and Aspirations in Making Good Data

We thought data to be simply given, but reality tells otherwise; it is c...

Please sign up or login with your details

Forgot password? Click here to reset