Improving Federated Learning Personalization via Model Agnostic Meta Learning

09/27/2019
by   Yihan Jiang, et al.
0

Federated Learning (FL) refers to learning a high quality global model based on decentralized data storage, without ever copying the raw data. A natural scenario arises with data created on mobile phones by the activity of their users. Given the typical data heterogeneity in such situations, it is natural to ask how can the global model be personalized for every such device, individually. In this work, we point out that the setting of Model Agnostic Meta Learning (MAML), where one optimizes for a fast, gradient-based, few-shot adaptation to a heterogeneous distribution of tasks, has a number of similarities with the objective of personalization for FL. We present FL as a natural source of practical applications for MAML algorithms, and make the following observations. 1) The popular FL algorithm, Federated Averaging, can be interpreted as a meta learning algorithm. 2) Careful fine-tuning can yield a global model with higher accuracy, which is at the same time easier to personalize. However, solely optimizing for the global model accuracy yields a weaker personalization result. 3) A model trained using a standard datacenter optimization method is much harder to personalize, compared to one trained using Federated Averaging, supporting the first claim. These results raise new questions for FL, MAML, and broader ML research.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/21/2021

Compositional Federated Learning: Applications in Distributionally Robust Averaging and Meta Learning

In the paper, we propose an effective and efficient Compositional Federa...
research
03/23/2023

Automated Federated Learning in Mobile Edge Networks – Fast Adaptation and Convergence

Federated Learning (FL) can be used in mobile edge networks to train mac...
research
04/11/2023

TinyReptile: TinyML with Federated Meta-Learning

Tiny machine learning (TinyML) is a rapidly growing field aiming to demo...
research
10/03/2022

PersA-FL: Personalized Asynchronous Federated Learning

We study the personalized federated learning problem under asynchronous ...
research
10/18/2019

Federated Learning with Unbiased Gradient Aggregation and Controllable Meta Updating

Federated Averaging (FedAvg) serves as the fundamental framework in Fede...
research
05/31/2021

Meta-HAR: Federated Representation Learning for Human Activity Recognition

Human activity recognition (HAR) based on mobile sensors plays an import...
research
05/31/2020

Global Convergence of MAML for LQR

The paper studies the performance of the Model-Agnostic Meta-Learning (M...

Please sign up or login with your details

Forgot password? Click here to reset