Federated Continual Learning through distillation in pervasive computing

07/17/2022
by   Anastasiia Usmanova, et al.
0

Federated Learning has been introduced as a new machine learning paradigm enhancing the use of local devices. At a server level, FL regularly aggregates models learned locally on distributed clients to obtain a more general model. Current solutions rely on the availability of large amounts of stored data at the client side in order to fine-tune the models sent by the server. Such setting is not realistic in mobile pervasive computing where data storage must be kept low and data characteristic can change dramatically. To account for this variability, a solution is to use the data regularly collected by the client to progressively adapt the received model. But such naive approach exposes clients to the well-known problem of catastrophic forgetting. To address this problem, we have defined a Federated Continual Learning approach which is mainly based on distillation. Our approach allows a better use of resources, eliminating the need to retrain from scratch at the arrival of new data and reducing memory usage by limiting the amount of data to be stored. This proposal has been evaluated in the Human Activity Recognition (HAR) domain and has shown to effectively reduce the catastrophic forgetting effect.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/17/2022

Federated Learning and catastrophic forgetting in pervasive computing: demonstration in HAR domain

Federated Learning has been introduced as a new machine learning paradig...
research
09/09/2021

A distillation-based approach integrating continual learning and federated learning for pervasive services

Federated Learning, a new machine learning paradigm enhancing the use of...
research
06/16/2023

HePCo: Data-Free Heterogeneous Prompt Consolidation for Continual Federated Learning

In this paper, we focus on the important yet understudied problem of Con...
research
09/01/2023

Jointly Exploring Client Drift and Catastrophic Forgetting in Dynamic Learning

Federated and Continual Learning have emerged as potential paradigms for...
research
07/02/2023

Don't Memorize; Mimic The Past: Federated Class Incremental Learning Without Episodic Memory

Deep learning models are prone to forgetting information learned in the ...
research
04/26/2021

Continual Distributed Learning for Crisis Management

Social media platforms such as Twitter provide an excellent resource for...
research
09/01/2021

Federated Reconnaissance: Efficient, Distributed, Class-Incremental Learning

We describe federated reconnaissance, a class of learning problems in wh...

Please sign up or login with your details

Forgot password? Click here to reset