Private Multi-Task Learning: Formulation and Applications to Federated Learning

08/30/2021
by   Shengyuan Hu, et al.
0

Many problems in machine learning rely on multi-task learning (MTL), in which the goal is to solve multiple related machine learning tasks simultaneously. MTL is particularly relevant for privacy-sensitive applications in areas such as healthcare, finance, and IoT computing, where sensitive data from multiple, varied sources are shared for the purpose of learning. In this work, we formalize notions of task-level privacy for MTL via joint differential privacy(JDP), a relaxation of differential privacy for mechanism design and distributed optimization. We then propose an algorithm for mean-regularized MTL, an objective commonly used for applications in personalized federated learning, subject to JDP. We analyze our objective and solver, providing certifiable guarantees on both privacy and utility. Empirically, we find that our method allows for improved privacy/utility trade-offs relative to global baselines across common federated learning benchmarks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/22/2019

Federated Learning with Bayesian Differential Privacy

We consider the problem of reinforcing federated learning with formal pr...
research
02/21/2020

Anonymizing Data for Privacy-Preserving Federated Learning

Federated learning enables training a global machine learning model from...
research
02/12/2022

Private Adaptive Optimization with Side Information

Adaptive optimization methods have become the default solvers for many m...
research
07/26/2021

Preliminary Steps Towards Federated Sentiment Classification

Automatically mining sentiment tendency contained in natural language is...
research
09/12/2019

Differentially Private Meta-Learning

Parameter-transfer is a well-known and versatile approach for meta-learn...
research
07/16/2022

Sotto Voce: Federated Speech Recognition with Differential Privacy Guarantees

Speech data is expensive to collect, and incredibly sensitive to its sou...
research
08/06/2020

Improving on-device speaker verification using federated learning with privacy

Information on speaker characteristics can be useful as side information...

Please sign up or login with your details

Forgot password? Click here to reset