Federated Multi-Task Learning under a Mixture of Distributions

08/23/2021
by   Othmane Marfoq, et al.
14

The increasing size of data generated by smartphones and IoT devices motivated the development of Federated Learning (FL), a framework for on-device collaborative training of machine learning models. First efforts in FL focused on learning a single global model with good average performance across clients, but the global model may be arbitrarily bad for a given client, due to the inherent heterogeneity of local data distributions. Federated multi-task learning (MTL) approaches can learn personalized models by formulating an opportune penalized optimization problem. The penalization term can capture complex relations among personalized models, but eschews clear statistical assumptions about local data distributions. In this work, we propose to study federated MTL under the flexible assumption that each local data distribution is a mixture of unknown underlying distributions. This assumption encompasses most of the existing personalized FL approaches and leads to federated EM-like algorithms for both client-server and fully decentralized settings. Moreover, it provides a principled way to serve personalized models to clients not seen at training time. The algorithms' convergence is analyzed through a novel federated surrogate optimization framework, which can be of general interest. Experimental results on FL benchmarks show that in most cases our approach provides models with higher accuracy and fairness than state-of-the-art methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/15/2021

Adapt to Adaptation: Learning Personalization for Cross-Silo Federated Learning

The goal of conventional federated learning (FL) is to train a global mo...
research
12/15/2020

Personalized Federated Learning with First Order Model Optimization

While federated learning traditionally aims to train a single global mod...
research
11/15/2022

Personalized Federated Learning with Multi-branch Architecture

Federated learning (FL) is a decentralized machine learning technique th...
research
03/12/2021

SCEI: A Smart-Contract Driven Edge Intelligence Framework for IoT Systems

Federated learning (FL) utilizes edge computing devices to collaborative...
research
04/26/2021

Communication-Efficient and Personalized Federated Lottery Ticket Learning

The lottery ticket hypothesis (LTH) claims that a deep neural network (i...
research
11/14/2022

FedCL: Federated Multi-Phase Curriculum Learning to Synchronously Correlate User Heterogeneity

Federated Learning (FL) is a new decentralized learning used for trainin...
research
06/24/2021

Personalized Federated Learning with Clustered Generalization

We study the recent emerging personalized federated learning (PFL) that ...

Please sign up or login with your details

Forgot password? Click here to reset