DeepAI
Log In Sign Up

Compositional Federated Learning: Applications in Distributionally Robust Averaging and Meta Learning

06/21/2021
by   Feihu Huang, et al.
7

In the paper, we propose an effective and efficient Compositional Federated Learning (ComFedL) algorithm for solving a new compositional Federated Learning (FL) framework, which frequently appears in many machine learning problems with a hierarchical structure such as distributionally robust federated learning and model-agnostic meta learning (MAML). Moreover, we study the convergence analysis of our ComFedL algorithm under some mild conditions, and prove that it achieves a fast convergence rate of O(1/√(T)), where T denotes the number of iteration. To the best of our knowledge, our algorithm is the first work to bridge federated learning with composition stochastic optimization. In particular, we first transform the distributionally robust FL (i.e., a minimax optimization problem) into a simple composition optimization problem by using KL divergence regularization. At the same time, we also first transform the distribution-agnostic MAML problem (i.e., a minimax optimization problem) into a simple composition optimization problem. Finally, we apply two popular machine learning tasks, i.e., distributionally robust FL and MAML to demonstrate the effectiveness of our algorithm.

READ FULL TEXT

page 1

page 2

page 3

page 4

09/27/2019

Improving Federated Learning Personalization via Model Agnostic Meta Learning

Federated Learning (FL) refers to learning a high quality global model b...
06/12/2022

Federated Learning on Riemannian Manifolds

Federated learning (FL) has found many important applications in smart-p...
06/29/2021

Achieving Statistical Optimality of Federated Learning: Beyond Stationary Points

Federated Learning (FL) is a promising framework that has great potentia...
05/04/2022

FEDNEST: Federated Bilevel, Minimax, and Compositional Optimization

Standard federated optimization methods successfully apply to stochastic...
06/09/2021

Memory-based Optimization Methods for Model-Agnostic Meta-Learning

Recently, model-agnostic meta-learning (MAML) has garnered tremendous at...
07/17/2022

Fast Composite Optimization and Statistical Recovery in Federated Learning

As a prevalent distributed learning paradigm, Federated Learning (FL) tr...
09/03/2021

Ground-Assisted Federated Learning in LEO Satellite Constellations

In Low Earth Orbit (LEO) mega constellations, there are relevant use cas...