Deep Class Incremental Learning from Decentralized Data

03/11/2022
by   Xiaohan Zhang, et al.
0

In this paper, we focus on a new and challenging decentralized machine learning paradigm in which there are continuous inflows of data to be addressed and the data are stored in multiple repositories. We initiate the study of data decentralized class-incremental learning (DCIL) by making the following contributions. Firstly, we formulate the DCIL problem and develop the experimental protocol. Secondly, we introduce a paradigm to create a basic decentralized counterpart of typical (centralized) class-incremental learning approaches, and as a result, establish a benchmark for the DCIL study. Thirdly, we further propose a Decentralized Composite knowledge Incremental Distillation framework (DCID) to transfer knowledge from historical models and multiple local sites to the general model continually. DCID consists of three main components namely local class-incremental learning, collaborated knowledge distillation among local models, and aggregated knowledge distillation from local models to the general one. We comprehensively investigate our DCID framework by using different implementations of the three components. Extensive experimental results demonstrate the effectiveness of our DCID framework. The codes of the baseline methods and the proposed DCIL will be released at https://github.com/zxxxxh/DCIL.

READ FULL TEXT

page 1

page 12

research
03/21/2022

Document-Level Relation Extraction with Adaptive Focal Loss and Knowledge Distillation

Document-level Relation Extraction (DocRE) is a more challenging task co...
research
07/18/2023

Class-relation Knowledge Distillation for Novel Class Discovery

We tackle the problem of novel class discovery, which aims to learn nove...
research
11/11/2020

Real-Time Decentralized knowledge Transfer at the Edge

Proliferation of edge networks creates islands of learning agents workin...
research
07/08/2018

Revisiting Distillation and Incremental Classifier Learning

One of the key differences between the learning mechanism of humans and ...
research
09/01/2022

A New Knowledge Distillation Network for Incremental Few-Shot Surface Defect Detection

Surface defect detection is one of the most essential processes for indu...
research
05/27/2022

A Decentralized Collaborative Learning Framework Across Heterogeneous Devices for Personalized Predictive Analytics

In this paper, we propose a Similarity-based Decentralized Knowledge Dis...
research
05/25/2023

Camera-Incremental Object Re-Identification with Identity Knowledge Evolution

Object Re-identification (ReID) aims to retrieve the probe object from m...

Please sign up or login with your details

Forgot password? Click here to reset