Gradual Federated Learning with Simulated Annealing

10/11/2021
by   Luong Trung Nguyen, et al.
0

Federated averaging (FedAvg) is a popular federated learning (FL) technique that updates the global model by averaging local models and then transmits the updated global model to devices for their local model update. One main limitation of FedAvg is that the average-based global model is not necessarily better than local models in the early stage of the training process so that FedAvg might diverge in realistic scenarios, especially when the data is non-identically distributed across devices and the number of data samples varies significantly from device to device. In this paper, we propose a new FL technique based on simulated annealing. The key idea of the proposed technique, henceforth referred to as simulated annealing-based FL (SAFL), is to allow a device to choose its local model when the global model is immature. Specifically, by exploiting the simulated annealing strategy, we make each device choose its local model with high probability in early iterations when the global model is immature. From extensive numerical experiments using various benchmark datasets, we demonstrate that SAFL outperforms the conventional FedAvg technique in terms of the convergence speed and the classification accuracy.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

02/23/2022

FedCAT: Towards Accurate Federated Learning via Device Concatenation

As a promising distributed machine learning paradigm, Federated Learning...
06/08/2021

Fast Federated Learning in the Presence of Arbitrary Device Unavailability

Federated Learning (FL) coordinates with numerous heterogeneous devices ...
08/16/2019

Federated Learning with Additional Mechanisms on Clients to Reduce Communication Costs

Federated learning (FL) enables on-device training over distributed netw...
11/29/2021

FedHM: Efficient Federated Learning for Heterogeneous Models via Low-rank Factorization

The underlying assumption of recent federated learning (FL) paradigms is...
02/21/2022

Backdoor Defense in Federated Learning Using Differential Testing and Outlier Detection

The goal of federated learning (FL) is to train one global model by aggr...
06/19/2020

FedFMC: Sequential Efficient Federated Learning on Non-iid Data

As a mechanism for devices to update a global model without sharing data...
09/11/2021

On the Initial Behavior Monitoring Issues in Federated Learning

In Federated Learning (FL), a group of workers participate to build a gl...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.