Networked Federated Multi-Task Learning

05/26/2021
by   Yasmin SarcheshmehPour, et al.
0

Many important application domains generate distributed collections of heterogeneous local datasets. These local datasets are often related via an intrinsic network structure that arises from domain-specific notions of similarity between local datasets. Different notions of similarity are induced by spatiotemporal proximity, statistical dependencies, or functional relations. We use this network structure to adaptively pool similar local datasets into nearly homogenous training sets for learning tailored models. Our main conceptual contribution is to formulate networked federated learning using the concept of generalized total variation (GTV) minimization as a regularizer. This formulation is highly flexible and can be combined with almost any parametric model including Lasso or deep neural networks. We unify and considerably extend some well-known approaches to federated multi-task learning. Our main algorithmic contribution is a novel federated learning algorithm that is well suited for distributed computing environments such as edge computing over wireless networks. This algorithm is robust against model misspecification and numerical errors arising from limited computational resources including processing time or wireless channel bandwidth. As our main technical contribution, we offer precise conditions on the local models as well on their network structure such that our algorithm learns nearly optimal local models. Our analysis reveals an interesting interplay between the (information-) geometry of local models and the (cluster-) geometry of their network.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/27/2020

Federated Learning From Big Data Over Networks

This paper formulates and studies a novel algorithm for federated learni...
research
02/08/2023

Towards Model-Agnostic Federated Learning over Networks

We present a model-agnostic federated learning method for decentralized ...
research
07/17/2022

Multi-Task and Transfer Learning for Federated Learning Applications

Federated learning enables many applications benefiting distributed and ...
research
06/30/2023

FedBone: Towards Large-Scale Federated Multi-Task Learning

Heterogeneous federated multi-task learning (HFMTL) is a federated learn...
research
02/14/2021

FedU: A Unified Framework for Federated Multi-Task Learning with Laplacian Regularization

Federated multi-task learning (FMTL) has emerged as a natural choice to ...
research
12/08/2020

Federated Multi-Task Learning for Competing Constraints

In addition to accuracy, fairness and robustness are two critical concer...

Please sign up or login with your details

Forgot password? Click here to reset