Differentially Private Meta-Learning

09/12/2019
by   Jeffrey Li, et al.
5

Parameter-transfer is a well-known and versatile approach for meta-learning, with applications including few-shot learning, federated learning, and reinforcement learning. However, parameter-transfer algorithms often require sharing models that have been trained on the samples from specific tasks, thus leaving the task-owners susceptible to breaches of privacy. We conduct the first formal study of privacy in this setting and formalize the notion of task-global differential privacy as a practical relaxation of more commonly studied threat models. We then propose a new differentially private algorithm for gradient-based parameter transfer that not only satisfies this privacy requirement but also retains provable transfer learning guarantees in convex settings. Empirically, we apply our analysis to the problem of federated learning with personalization and show that allowing the relaxation to task-global privacy from the more commonly studied notion of local privacy leads to dramatically increased performance in recurrent neural language modeling.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/08/2021

DiPSeN: Differentially Private Self-normalizing Neural Networks For Adversarial Robustness in Federated Learning

The need for robust, secure and private machine learning is an important...
research
12/13/2021

Efficient Differentially Private Secure Aggregation for Federated Learning via Hardness of Learning with Errors

Federated machine learning leverages edge computing to develop models fr...
research
08/30/2021

Private Multi-Task Learning: Formulation and Applications to Federated Learning

Many problems in machine learning rely on multi-task learning (MTL), in ...
research
11/15/2019

Generative Models for Effective ML on Private, Decentralized Datasets

To improve real-world applications of machine learning, experienced mode...
research
08/02/2022

Differentially Private Vertical Federated Clustering

In many applications, multiple parties have private data regarding the s...
research
12/20/2017

Differentially Private Distributed Learning for Language Modeling Tasks

One of the big challenges in machine learning applications is that train...
research
11/14/2020

A Theoretical Perspective on Differentially Private Federated Multi-task Learning

In the era of big data, the need to expand the amount of data through da...

Please sign up or login with your details

Forgot password? Click here to reset