Limits of Private Learning with Access to Public Data

10/25/2019
by   Noga Alon, et al.
10

We consider learning problems where the training set consists of two types of examples: private and public. The goal is to design a learning algorithm that satisfies differential privacy only with respect to the private examples. This setting interpolates between private learning (where all examples are private) and classical learning (where all examples are public). We study the limits of learning in this setting in terms of private and public sample complexities. We show that any hypothesis class of VC-dimension d can be agnostically learned up to an excess error of α using only (roughly) d/α public examples and d/α^2 private labeled examples. This result holds even when the public examples are unlabeled. This gives a quadratic improvement over the standard d/α^2 upper bound on the public sample complexity (where private examples can be ignored altogether if the public examples are labeled). Furthermore, we give a nearly matching lower bound, which we prove via a generic reduction from this setting to the one of private learning without public data.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/23/2020

Private Query Release Assisted by Public Data

We study the problem of differentially private query release assisted by...
research
08/01/2020

Learning from Mixtures of Private and Public Populations

We initiate the study of a new model of supervised learning under privac...
research
08/11/2023

Private Distribution Learning with Public Data: The View from Sample Compression

We study the problem of private distribution learning with access to pub...
research
09/17/2022

On PAC Learning Halfspaces in Non-interactive Local Privacy Model with Public Unlabeled Data

In this paper, we study the problem of PAC learning halfspaces in the no...
research
06/06/2023

PILLAR: How to make semi-private learning more effective

In Semi-Supervised Semi-Private (SP) learning, the learner has access to...
research
07/31/2019

Career Choice as an Extended Spatial Evolutionary Public Goods Game

We propose an extended spatial evolutionary public goods game (SEPGG) mo...
research
11/24/2020

InstaHide's Sample Complexity When Mixing Two Private Images

Inspired by InstaHide challenge [Huang, Song, Li and Arora'20], [Chen, S...

Please sign up or login with your details

Forgot password? Click here to reset