Semi-Synchronous Federated Learning

by   Dimitris Stripelis, et al.

There are situations where data relevant to a machine learning problem are distributed among multiple locations that cannot share the data due to regulatory, competitiveness, or privacy reasons. For example, data present in users' cellphones, manufacturing data of companies in a given industrial sector, or medical records located at different hospitals. Federated Learning (FL) provides an approach to learn a joint model over all the available data across silos. In many cases, participating sites have different data distributions and computational capabilities. In these heterogeneous environments previous approaches exhibit poor performance: synchronous FL protocols are communication efficient, but have slow learning convergence; conversely, asynchronous FL protocols have faster convergence, but at a higher communication cost. Here we introduce a novel Semi-Synchronous Federated Learning protocol that mixes local models periodically with minimal idle time and fast convergence. We show through extensive experiments that our approach significantly outperforms previous work in data and computationally heterogeneous environments.



page 4

page 15


Accelerating Federated Learning in Heterogeneous Data and Computational Environments

There are situations where data relevant to a machine learning problem a...

Scaling Neuroscience Research using Federated Learning

The amount of biomedical data continues to grow rapidly. However, the ab...

SAFA: a Semi-Asynchronous Protocol for Fast Federated Learning with Low Overhead

Federated learning (FL) has attracted increasing attention as a promisin...

Secure Aggregation for Buffered Asynchronous Federated Learning

Federated learning (FL) typically relies on synchronous training, which ...

Personalized Federated Learning of Driver Prediction Models for Autonomous Driving

Autonomous vehicles (AVs) must interact with a diverse set of human driv...

FedMAX: Mitigating Activation Divergence for Accurate and Communication-Efficient Federated Learning

In this paper, we identify a new phenomenon called activation-divergence...

Ground-Assisted Federated Learning in LEO Satellite Constellations

In Low Earth Orbit (LEO) mega constellations, there are relevant use cas...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.