Semi-Synchronous Federated Learning

02/04/2021
by   Dimitris Stripelis, et al.
0

There are situations where data relevant to a machine learning problem are distributed among multiple locations that cannot share the data due to regulatory, competitiveness, or privacy reasons. For example, data present in users' cellphones, manufacturing data of companies in a given industrial sector, or medical records located at different hospitals. Federated Learning (FL) provides an approach to learn a joint model over all the available data across silos. In many cases, participating sites have different data distributions and computational capabilities. In these heterogeneous environments previous approaches exhibit poor performance: synchronous FL protocols are communication efficient, but have slow learning convergence; conversely, asynchronous FL protocols have faster convergence, but at a higher communication cost. Here we introduce a novel Semi-Synchronous Federated Learning protocol that mixes local models periodically with minimal idle time and fast convergence. We show through extensive experiments that our approach significantly outperforms previous work in data and computationally heterogeneous environments.

READ FULL TEXT

Authors

page 4

page 15

08/25/2020

Accelerating Federated Learning in Heterogeneous Data and Computational Environments

There are situations where data relevant to a machine learning problem a...
02/16/2021

Scaling Neuroscience Research using Federated Learning

The amount of biomedical data continues to grow rapidly. However, the ab...
10/03/2019

SAFA: a Semi-Asynchronous Protocol for Fast Federated Learning with Low Overhead

Federated learning (FL) has attracted increasing attention as a promisin...
10/05/2021

Secure Aggregation for Buffered Asynchronous Federated Learning

Federated learning (FL) typically relies on synchronous training, which ...
12/02/2021

Personalized Federated Learning of Driver Prediction Models for Autonomous Driving

Autonomous vehicles (AVs) must interact with a diverse set of human driv...
04/07/2020

FedMAX: Mitigating Activation Divergence for Accurate and Communication-Efficient Federated Learning

In this paper, we identify a new phenomenon called activation-divergence...
09/03/2021

Ground-Assisted Federated Learning in LEO Satellite Constellations

In Low Earth Orbit (LEO) mega constellations, there are relevant use cas...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.