Complexity-aware Adaptive Training and Inference for Edge-Cloud Distributed AI Systems

09/14/2021
by   Yinghan Long, et al.
0

The ubiquitous use of IoT and machine learning applications is creating large amounts of data that require accurate and real-time processing. Although edge-based smart data processing can be enabled by deploying pretrained models, the energy and memory constraints of edge devices necessitate distributed deep learning between the edge and the cloud for complex data. In this paper, we propose a distributed AI system to exploit both the edge and the cloud for training and inference. We propose a new architecture, MEANet, with a main block, an extension block, and an adaptive block for the edge. The inference process can terminate at either the main block, the extension block, or the cloud. The MEANet is trained to categorize inputs into easy/hard/complex classes. The main block identifies instances of easy/hard classes and classifies easy classes with high confidence. Only data with high probabilities of belonging to hard classes would be sent to the extension block for prediction. Further, only if the neural network at the edge shows low confidence in the prediction, the instance is considered complex and sent to the cloud for further processing. The training technique lends to the majority of inference on edge devices while going to the cloud only for a small set of complex jobs, as determined by the edge. The performance of the proposed system is evaluated via extensive experiments using modified models of ResNets and MobileNetV2 on CIFAR-100 and ImageNet datasets. The results show that the proposed distributed model has improved accuracy and energy consumption, indicating its capacity to adapt.

READ FULL TEXT
research
01/06/2022

A Framework for Energy-aware Evaluation of Distributed Data Processing Platforms in Edge-Cloud Environment

Distributed data processing platforms (e.g., Hadoop, Spark, and Flink) a...
research
05/21/2020

Conditionally Deep Hybrid Neural Networks Across Edge and Cloud

The pervasiveness of "Internet-of-Things" in our daily life has led to a...
research
07/21/2023

Adaptive ResNet Architecture for Distributed Inference in Resource-Constrained IoT Systems

As deep neural networks continue to expand and become more complex, most...
research
04/07/2022

Energy-Efficient Adaptive Machine Learning on IoT End-Nodes With Class-Dependent Confidence

Energy-efficient machine learning models that can run directly on edge d...
research
04/14/2020

Budget Learning via Bracketing

Conventional machine learning applications in the mobile/IoT setting tra...
research
08/31/2023

An Energy-Aware Approach to Design Self-Adaptive AI-based Applications on the Edge

The advent of edge devices dedicated to machine learning tasks enabled t...
research
06/25/2021

EARLIN: Early Out-of-Distribution Detection for Resource-efficient Collaborative Inference

Collaborative inference enables resource-constrained edge devices to mak...

Please sign up or login with your details

Forgot password? Click here to reset