Pisces: Efficient Federated Learning via Guided Asynchronous Training

06/18/2022
by   Zhifeng Jiang, et al.
0

Federated learning (FL) is typically performed in a synchronous parallel manner, where the involvement of a slow client delays a training iteration. Current FL systems employ a participant selection strategy to select fast clients with quality data in each iteration. However, this is not always possible in practice, and the selection strategy often has to navigate an unpleasant trade-off between the speed and the data quality of clients. In this paper, we present Pisces, an asynchronous FL system with intelligent participant selection and model aggregation for accelerated training. To avoid incurring excessive resource cost and stale training computation, Pisces uses a novel scoring mechanism to identify suitable clients to participate in a training iteration. It also adapts the pace of model aggregation to dynamically bound the progress gap between the selected clients and the server, with a provable convergence guarantee in a smooth non-convex setting. We have implemented Pisces in an open-source FL platform called Plato, and evaluated its performance in large-scale experiments with popular vision and language models. Pisces outperforms the state-of-the-art synchronous and asynchronous schemes, accelerating the time-to-accuracy by up to 2.0x and 1.9x, respectively.

READ FULL TEXT
06/11/2021

Federated Learning with Buffered Asynchronous Aggregation

Federated Learning (FL) trains a shared model across distributed devices...
11/08/2021

Papaya: Practical, Private, and Scalable Federated Learning

Cross-device Federated Learning (FL) is a distributed learning paradigm ...
12/15/2021

Blockchain-enabled Server-less Federated Learning

Motivated by the heterogeneous nature of devices participating in large-...
07/12/2020

VAFL: a Method of Vertical Asynchronous Federated Learning

Horizontal Federated learning (FL) handles multi-client data that share ...
11/17/2020

Stochastic Client Selection for Federated Learning with Volatile Clients

Federated Learning (FL), arising as a novel secure learning paradigm, ha...
05/12/2021

Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learning

This work addresses the problem of optimizing communications between ser...
10/03/2019

SAFA: a Semi-Asynchronous Protocol for Fast Federated Learning with Low Overhead

Federated learning (FL) has attracted increasing attention as a promisin...