Global Convergence of Federated Learning for Mixed Regression

06/15/2022
by   Lili Su, et al.
0

This paper studies the problem of model training under Federated Learning when clients exhibit cluster structure. We contextualize this problem in mixed regression, where each client has limited local data generated from one of k unknown regression models. We design an algorithm that achieves global convergence from any initialization, and works even when local data volume is highly unbalanced – there could exist clients that contain O(1) data points only. Our algorithm first runs moment descent on a few anchor clients (each with Ω̃(k) data points) to obtain coarse model estimates. Then each client alternately estimates its cluster labels and refines the model estimates based on FedAvg or FedProx. A key innovation in our analysis is a uniform estimate on the clustering errors, which we prove by bounding the VC dimension of general polynomial concept classes based on the theory of algebraic geometry.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/12/2022

Find Your Friends: Personalized Federated Learning with the Right Collaborators

In the traditional federated learning setting, a central server coordina...
research
11/05/2021

Data Selection for Efficient Model Update in Federated Learning

The Federated Learning workflow of training a centralized model with dis...
research
02/13/2023

FilFL: Accelerating Federated Learning via Client Filtering

Federated learning is an emerging machine learning paradigm that enables...
research
08/25/2023

Heterogeneous Federated Learning via Personalized Generative Networks

Federated Learning (FL) allows several clients to construct a common glo...
research
07/23/2023

Scalable solution to crossed random effects model with random slopes

The crossed random-effects model is widely used in applied statistics, f...
research
11/12/2020

Coded Computing for Low-Latency Federated Learning over Wireless Edge Networks

Federated learning enables training a global model from data located at ...
research
05/01/2023

Scalable Data Point Valuation in Decentralized Learning

Existing research on data valuation in federated and swarm learning focu...

Please sign up or login with your details

Forgot password? Click here to reset