An Efficient Split Fine-tuning Framework for Edge and Cloud Collaborative Learning

11/30/2022
by   Shaohuai Shi, et al.
0

To enable the pre-trained models to be fine-tuned with local data on edge devices without sharing data with the cloud, we design an efficient split fine-tuning (SFT) framework for edge and cloud collaborative learning. We propose three novel techniques in this framework. First, we propose a matrix decomposition-based method to compress the intermediate output of a neural network to reduce the communication volume between the edge device and the cloud server. Second, we eliminate particular links in the model without affecting the convergence performance in fine-tuning. Third, we implement our system atop PyTorch to allow users to easily extend their existing training scripts to enjoy the efficient edge and cloud collaborative learning. Experiments results on 9 NLP datasets show that our framework can reduce the communication traffic by 96 times with little impact on the model accuracy.

READ FULL TEXT
research
09/12/2022

DUET: A Tuning-Free Device-Cloud Collaborative Parameters Generation Framework for Efficient Device Model Generalization

Device Model Generalization (DMG) is a practical yet under-investigated ...
research
04/14/2021

Towards Unsupervised Fine-Tuning for Edge Video Analytics

Judging by popular and generic computer vision challenges, such as the I...
research
10/21/2022

On-Device Model Fine-Tuning with Label Correction in Recommender Systems

To meet the practical requirements of low latency, low cost, and good pr...
research
10/18/2020

Characterizing and Taming Model Instability Across Edge Devices

The same machine learning model running on different edge devices may pr...
research
08/30/2021

Auto-Split: A General Framework of Collaborative Edge-Cloud AI

In many industry scale applications, large and resource consuming machin...
research
05/04/2020

CDC: Classification Driven Compression for Bandwidth Efficient Edge-Cloud Collaborative Deep Learning

The emerging edge-cloud collaborative Deep Learning (DL) paradigm aims a...
research
06/25/2021

EARLIN: Early Out-of-Distribution Detection for Resource-efficient Collaborative Inference

Collaborative inference enables resource-constrained edge devices to mak...

Please sign up or login with your details

Forgot password? Click here to reset