Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing

10/04/2019
by   En Li, et al.
0

As a key technology of enabling Artificial Intelligence (AI) applications in 5G era, Deep Neural Networks (DNNs) have quickly attracted widespread attention. However, it is challenging to run computation-intensive DNN-based tasks on mobile devices due to the limited computation resources. What's worse, traditional cloud-assisted DNN inference is heavily hindered by the significant wide-area network latency, leading to poor real-time performance as well as low quality of user experience. To address these challenges, in this paper, we propose Edgent, a framework that leverages edge computing for DNN collaborative inference through device-edge synergy. Edgent exploits two design knobs: (1) DNN partitioning that adaptively partitions computation between device and edge for purpose of coordinating the powerful cloud resource and the proximal edge resource for real-time DNN inference; (2) DNN right-sizing that further reduces computing latency via early exiting inference at an appropriate intermediate DNN layer. In addition, considering the potential network fluctuation in real-world deployment, Edgentis properly design to specialize for both static and dynamic network environment. Specifically, in a static environment where the bandwidth changes slowly, Edgent derives the best configurations with the assist of regression-based prediction models, while in a dynamic environment where the bandwidth varies dramatically, Edgent generates the best execution plan through the online change point detection algorithm that maps the current bandwidth state to the optimal configuration. We implement Edgent prototype based on the Raspberry Pi and the desktop PC and the extensive experimental evaluations demonstrate Edgent's effectiveness in enabling on-demand low-latency edge intelligence.

READ FULL TEXT
research
06/20/2018

Edge Intelligence: On-Demand Deep Learning Model Co-Inference with Device-Edge Synergy

As the backbone technology of machine learning, deep neural networks (DN...
research
06/11/2020

Real-Time Video Inference on Edge Devices via Adaptive Model Streaming

Real-time video inference on compute-limited edge devices like mobile ph...
research
05/05/2021

ScissionLite: Accelerating Distributed Deep Neural Networks Using Transfer Layer

Industrial Internet of Things (IIoT) applications can benefit from lever...
research
12/06/2020

CoEdge: Cooperative DNN Inference with Adaptive Workload Partitioning over Heterogeneous Edge Devices

Recent advances in artificial intelligence have driven increasing intell...
research
06/15/2022

Resource-Constrained Edge AI with Early Exit Prediction

By leveraging the data sample diversity, the early-exit network recently...
research
11/13/2021

A Framework for Routing DNN Inference Jobs over Distributed Computing Networks

Ubiquitous artificial intelligence (AI) is considered one of the key ser...
research
06/21/2023

Adaptive DNN Surgery for Selfish Inference Acceleration with On-demand Edge Resource

Deep Neural Networks (DNNs) have significantly improved the accuracy of ...

Please sign up or login with your details

Forgot password? Click here to reset