Accelerating Federated Learning in Heterogeneous Data and Computational Environments

08/25/2020
by   Dimitris Stripelis, et al.
0

There are situations where data relevant to a machine learning problem are distributed among multiple locations that cannot share the data due to regulatory, competitiveness, or privacy reasons. For example, data present in users' cellphones, manufacturing data of companies in a given industrial sector, or medical records located at different hospitals. Moreover, participating sites often have different data distributions and computational capabilities. Federated Learning provides an approach to learn a joint model over all the available data in these environments. In this paper, we introduce a novel distributed validation weighting scheme (DVW), which evaluates the performance of a learner in the federation against a distributed validation set. Each learner reserves a small portion (e.g., 5 examples as a validation dataset and allows other learners models to be evaluated against it. We empirically show that DVW results in better performance compared to established methods, such as FedAvg, both under synchronous and asynchronous communication protocols in data and computationally heterogeneous environments.

READ FULL TEXT

Authors

page 11

02/04/2021

Semi-Synchronous Federated Learning

There are situations where data relevant to a machine learning problem a...
02/16/2021

Scaling Neuroscience Research using Federated Learning

The amount of biomedical data continues to grow rapidly. However, the ab...
11/16/2021

FedCostWAvg: A new averaging for better Federated Learning

We propose a simple new aggregation strategy for federated learning that...
03/22/2021

Real-time End-to-End Federated Learning: An Automotive Case Study

With the development and the increasing interests in ML/DL fields, compa...
08/17/2020

WAFFLE: Watermarking in Federated Learning

Creators of machine learning models can use watermarking as a technique ...
10/14/2019

Federated Learning for Coalition Operations

Machine Learning in coalition settings requires combining insights avail...
06/05/2022

Impossibility of Collective Intelligence

Democratization of AI involves training and deploying machine learning m...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.