Real-Time Decentralized knowledge Transfer at the Edge

11/11/2020
by   Orpaz Goldstein, et al.
0

Proliferation of edge networks creates islands of learning agents working on local streams of data. Transferring knowledge between these agents in real-time without exposing private data allows for collaboration to decrease learning time, and increase model confidence. Incorporating knowledge from data that was not seen by a local model creates an ability to debias a local model, or add to classification abilities on data never before seen. Transferring knowledge in a decentralized approach allows for models to retain their local insights, in turn allowing for local flavors of a machine learning model. This approach suits the decentralized architecture of edge networks, as a local edge node will serve a community of learning agents that will likely encounter similar data. We propose a method based on knowledge distillation for pairwise knowledge transfer pipelines, and compare to other popular knowledge transfer methods. Additionally, we test different scenarios of knowledge transfer network construction and show the practicality of our approach. Based on our experiments we show knowledge transfer using our model outperforms common methods in a real time transfer scenario.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/01/2020

Classification of Diabetic Retinopathy Using Unlabeled Data and Knowledge Distillation

Knowledge distillation allows transferring knowledge from a pre-trained ...
research
03/11/2022

Deep Class Incremental Learning from Decentralized Data

In this paper, we focus on a new and challenging decentralized machine l...
research
11/28/2022

Decentralized Learning with Multi-Headed Distillation

Decentralized learning with private data is a central problem in machine...
research
04/09/2023

Homogenizing Non-IID datasets via In-Distribution Knowledge Distillation for Decentralized Learning

Decentralized learning enables serverless training of deep neural networ...
research
07/29/2023

The effect of network topologies on fully decentralized learning: a preliminary investigation

In a decentralized machine learning system, data is typically partitione...
research
10/22/2017

Searching for effective and efficient way of knowledge transfer within an organization

In this paper three models of knowledge transfer in organization are con...
research
12/29/2018

A Two-Phase Dynamic Throughput Optimization Model for Big Data Transfers

The amount of data moved over dedicated and non-dedicated network links ...

Please sign up or login with your details

Forgot password? Click here to reset