Privacy-preserving Learning via Deep Net Pruning

03/04/2020
by   Yangsibo Huang, et al.
18

This paper attempts to answer the question whether neural network pruning can be used as a tool to achieve differential privacy without losing much data utility. As a first step towards understanding the relationship between neural network pruning and differential privacy, this paper proves that pruning a given layer of the neural network is equivalent to adding a certain amount of differentially private noise to its hidden-layer activations. The paper also presents experimental results to show the practical implications of the theoretical finding and the key parameter values in a simple practical setting. These results show that neural network pruning can be a more effective alternative to adding differentially private noise for neural networks.

READ FULL TEXT

page 2

page 10

page 11

page 40

page 41

page 42

research
11/29/2021

Architecture Matters: Investigating the Influence of Differential Privacy on Neural Network Design

One barrier to more widespread adoption of differentially private neural...
research
03/08/2023

Differential Privacy Meets Neural Network Pruning

A major challenge in applying differential privacy to training deep neur...
research
01/30/2019

Private Q-Learning with Functional Noise in Continuous Spaces

We consider privacy-preserving algorithms for deep reinforcement learnin...
research
06/07/2021

Photonic Differential Privacy with Direct Feedback Alignment

Optical Processing Units (OPUs) – low-power photonic chips dedicated to ...
research
07/29/2022

Content-Aware Differential Privacy with Conditional Invertible Neural Networks

Differential privacy (DP) has arisen as the gold standard in protecting ...
research
07/11/2018

Differentially Private False Discovery Rate Control

Differential privacy provides a rigorous framework for privacy-preservin...
research
05/24/2016

A note on privacy preserving iteratively reweighted least squares

Iteratively reweighted least squares (IRLS) is a widely-used method in m...

Please sign up or login with your details

Forgot password? Click here to reset