DeepAI AI Chat
Log In Sign Up

Federated Continual Learning for Text Classification via Selective Inter-client Transfer

by   Yatin Chaudhary, et al.

In this work, we combine the two paradigms: Federated Learning (FL) and Continual Learning (CL) for text classification task in cloud-edge continuum. The objective of Federated Continual Learning (FCL) is to improve deep learning models over life time at each client by (relevant and efficient) knowledge transfer without sharing data. Here, we address challenges in minimizing inter-client interference while knowledge sharing due to heterogeneous tasks across clients in FCL setup. In doing so, we propose a novel framework, Federated Selective Inter-client Transfer (FedSeIT) which selectively combines model parameters of foreign clients. To further maximize knowledge transfer, we assess domain overlap and select informative tasks from the sequence of historical tasks at each foreign client while preserving privacy. Evaluating against the baselines, we show improved performance, a gain of (average) 12.4% in text classification over a sequence of tasks using five datasets from diverse domains. To the best of our knowledge, this is the first work that applies FCL to NLP.


page 1

page 2

page 3

page 4


Federated Continual Learning with Adaptive Parameter Communication

There has been a surge of interest in continual learning and federated l...

Addressing Client Drift in Federated Continual Learning with Adaptive Optimization

Federated learning has been extensively studied and is the prevalent met...

FedKNOW: Federated Continual Learning with Signature Task Knowledge Integration at Edge

Deep Neural Networks (DNNs) have been ubiquitously adopted in internet o...

FedSpeech: Federated Text-to-Speech with Continual Learning

Federated learning enables collaborative training of machine learning mo...

Continual Learning for Text Classification with Information Disentanglement Based Regularization

Continual learning has become increasingly important as it enables NLP m...

Tackling Dynamics in Federated Incremental Learning with Variational Embedding Rehearsal

Federated Learning is a fast growing area of ML where the training datas...

Continual Horizontal Federated Learning for Heterogeneous Data

Federated learning is a promising machine learning technique that enable...