Evaluation and Optimization of Distributed Machine Learning Techniques for Internet of Things

03/03/2021
by   Yansong Gao, et al.
0

Federated learning (FL) and split learning (SL) are state-of-the-art distributed machine learning techniques to enable machine learning training without accessing raw data on clients or end devices. However, their comparative training performance under real-world resource-restricted Internet of Things (IoT) device settings, e.g., Raspberry Pi, remains barely studied, which, to our knowledge, have not yet been evaluated and compared, rendering inconvenient reference for practitioners. This work firstly provides empirical comparisons of FL and SL in real-world IoT settings regarding (i) learning performance with heterogeneous data distributions and (ii) on-device execution overhead. Our analyses in this work demonstrate that the learning performance of SL is better than FL under an imbalanced data distribution but worse than FL under an extreme non-IID data distribution. Recently, FL and SL are combined to form splitfed learning (SFL) to leverage each of their benefits (e.g., parallel training of FL and lightweight on-device computation requirement of SL). This work then considers FL, SL, and SFL, and mount them on Raspberry Pi devices to evaluate their performance, including training time, communication overhead, power consumption, and memory usage. Besides evaluations, we apply two optimizations. Firstly, we generalize SFL by carefully examining the possibility of a hybrid type of model training at the server-side. The generalized SFL merges sequential (dependent) and parallel (independent) processes of model training and is thus beneficial for a system with large-scaled IoT devices, specifically at the server-side operations. Secondly, we propose pragmatic techniques to substantially reduce the communication overhead by up to four times for the SL and (generalized) SFL.

READ FULL TEXT

page 8

page 9

page 10

research
03/30/2020

End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things

This work is the first attempt to evaluate and compare felderated learni...
research
10/26/2022

Low-latency Federated Learning with DNN Partition in Distributed Industrial IoT Networks

Federated Learning (FL) empowers Industrial Internet of Things (IIoT) wi...
research
05/31/2023

An Empirical Study of Federated Learning on IoT-Edge Devices: Resource Allocation and Heterogeneity

Nowadays, billions of phones, IoT and edge devices around the world gene...
research
08/25/2023

Federated Learning in IoT: a Survey from a Resource-Constrained Perspective

The IoT ecosystem is able to leverage vast amounts of data for intellige...
research
01/29/2022

Towards Fast and Accurate Federated Learning with non-IID Data for Cloud-Based IoT Applications

As a promising method of central model training on decentralized device ...
research
05/24/2022

FedEntropy: Efficient Device Grouping for Federated Learning Using Maximum Entropy Judgment

Along with the popularity of Artificial Intelligence (AI) and Internet-o...
research
07/25/2023

Federated Split Learning with Only Positive Labels for resource-constrained IoT environment

Distributed collaborative machine learning (DCML) is a promising method ...

Please sign up or login with your details

Forgot password? Click here to reset