Accelerating Federated Learning in Heterogeneous Data and Computational Environments

by   Dimitris Stripelis, et al.

There are situations where data relevant to a machine learning problem are distributed among multiple locations that cannot share the data due to regulatory, competitiveness, or privacy reasons. For example, data present in users' cellphones, manufacturing data of companies in a given industrial sector, or medical records located at different hospitals. Moreover, participating sites often have different data distributions and computational capabilities. Federated Learning provides an approach to learn a joint model over all the available data in these environments. In this paper, we introduce a novel distributed validation weighting scheme (DVW), which evaluates the performance of a learner in the federation against a distributed validation set. Each learner reserves a small portion (e.g., 5 examples as a validation dataset and allows other learners models to be evaluated against it. We empirically show that DVW results in better performance compared to established methods, such as FedAvg, both under synchronous and asynchronous communication protocols in data and computationally heterogeneous environments.



page 11


Semi-Synchronous Federated Learning

There are situations where data relevant to a machine learning problem a...

Scaling Neuroscience Research using Federated Learning

The amount of biomedical data continues to grow rapidly. However, the ab...

FedCostWAvg: A new averaging for better Federated Learning

We propose a simple new aggregation strategy for federated learning that...

Real-time End-to-End Federated Learning: An Automotive Case Study

With the development and the increasing interests in ML/DL fields, compa...

WAFFLE: Watermarking in Federated Learning

Creators of machine learning models can use watermarking as a technique ...

Federated Learning for Coalition Operations

Machine Learning in coalition settings requires combining insights avail...

Impossibility of Collective Intelligence

Democratization of AI involves training and deploying machine learning m...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.