Applied Federated Learning: Architectural Design for Robust and Efficient Learning in Privacy Aware Settings

06/02/2022
by   Branislav Stojkovic, et al.
0

The classical machine learning paradigm requires the aggregation of user data in a central location where machine learning practitioners can preprocess data, calculate features, tune models and evaluate performance. The advantage of this approach includes leveraging high performance hardware (such as GPUs) and the ability of machine learning practitioners to do in depth data analysis to improve model performance. However, these advantages may come at a cost to data privacy. User data is collected, aggregated, and stored on centralized servers for model development. Centralization of data poses risks, including a heightened risk of internal and external security incidents as well as accidental data misuse. Federated learning with differential privacy is designed to avoid the server-side centralization pitfall by bringing the ML learning step to users' devices. Learning is done in a federated manner where each mobile device runs a training loop on a local copy of a model. Updates from on-device models are sent to the server via encrypted communication and through differential privacy to improve the global model. In this paradigm, users' personal data remains on their devices. Surprisingly, model training in this manner comes at a fairly minimal degradation in model performance. However, federated learning comes with many other challenges due to its distributed nature, heterogeneous compute environments and lack of data visibility. This paper explores those challenges and outlines an architectural design solution we are exploring and testing to productionize federated learning at Meta scale.

READ FULL TEXT

page 1

page 2

page 3

page 4

10/19/2020

From Distributed Machine Learning To Federated Learning: In The View Of Data Privacy And Security

Federated learning is an improved version of distributed machine learnin...
05/11/2021

Federated Unbiased Learning to Rank

Unbiased Learning to Rank (ULTR) studies the problem of learning a ranki...
12/03/2018

Protection Against Reconstruction and Its Applications in Private Federated Learning

Federated learning has become an exciting direction for both research an...
05/05/2020

Information-Theoretic Bounds on the Generalization Error and Privacy Leakage in Federated Learning

Machine learning algorithms operating on mobile networks can be characte...
10/07/2019

Differential Privacy-enabled Federated Learning for Sensitive Health Data

Leveraging real-world health data for machine learning tasks requires ad...
03/31/2021

Federated Learning: A Signal Processing Perspective

The dramatic success of deep learning is largely due to the availability...
03/15/2022

Privacy-Aware Compression for Federated Data Analysis

Federated data analytics is a framework for distributed data analysis wh...