Dynamic Gradient Aggregation for Federated Domain Adaptation

06/14/2021
by   Dimitrios Dimitriadis, et al.
0

In this paper, a new learning algorithm for Federated Learning (FL) is introduced. The proposed scheme is based on a weighted gradient aggregation using two-step optimization to offer a flexible training pipeline. Herein, two different flavors of the aggregation method are presented, leading to an order of magnitude improvement in convergence speed compared to other distributed or FL training algorithms like BMUF and FedAvg. Further, the aggregation algorithm acts as a regularizer of the gradient quality. We investigate the effect of our FL algorithm in supervised and unsupervised Speech Recognition (SR) scenarios. The experimental validation is performed based on three tasks: first, the LibriSpeech task showing a speed-up of 7x and 6 (WERR) compared to the baseline results. The second task is based on session adaptation providing 20 unsupervised pipeline is applied to the conversational SR task. The proposed FL system outperforms the baseline systems in both convergence speed and overall model performance.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/06/2020

Federated Transfer Learning with Dynamic Gradient Aggregation

In this paper, a Federated Learning (FL) simulation platform is introduc...
research
08/20/2022

Just-in-Time Aggregation for Federated Learning

The increasing number and scale of federated learning (FL) jobs necessit...
research
09/18/2020

Federated Learning with Nesterov Accelerated Gradient Momentum Method

Federated learning (FL) is a fast-developing technique that allows multi...
research
07/05/2022

AVDDPG: Federated reinforcement learning applied to autonomous platoon control

Since 2016 federated learning (FL) has been an evolving topic of discuss...
research
09/21/2023

Enabling Quartile-based Estimated-Mean Gradient Aggregation As Baseline for Federated Image Classifications

Federated Learning (FL) has revolutionized how we train deep neural netw...
research
12/16/2021

CodedPaddedFL and CodedSecAgg: Straggler Mitigation and Secure Aggregation in Federated Learning

We present two novel coded federated learning (FL) schemes for linear re...
research
03/30/2022

Federated Domain Adaptation for ASR with Full Self-Supervision

Cross-device federated learning (FL) protects user privacy by collaborat...

Please sign up or login with your details

Forgot password? Click here to reset