DeepAI AI Chat
Log In Sign Up

Distributed Deep Learning Inference Acceleration using Seamless Collaboration in Edge Computing

by   Nan Li, et al.

This paper studies inference acceleration using distributed convolutional neural networks (CNNs) in collaborative edge computing. To ensure inference accuracy in inference task partitioning, we consider the receptive-field when performing segment-based partitioning. To maximize the parallelization between the communication and computing processes, thereby minimizing the total inference time of an inference task, we design a novel task collaboration scheme in which the overlapping zone of the sub-tasks on secondary edge servers (ESs) is executed on the host ES, named as HALP. We further extend HALP to the scenario of multiple tasks. Experimental results show that HALP can accelerate CNN inference in VGG-16 by 1.7-2.0x for a single task and 1.7-1.8x for 4 tasks per batch on GTX 1080TI and JETSON AGX Xavier, which outperforms the state-of-the-art work MoDNN. Moreover, we evaluate the service reliability under time-variant channel, which shows that HALP is an effective solution to ensure high service reliability with strict service deadline.


Receptive Field-based Segmentation for Distributed CNN Inference Acceleration in Collaborative Edge Computing

This paper studies inference acceleration using distributed convolutiona...

Design and Prototyping Distributed CNN Inference Acceleration in Edge Computing

For time-critical IoT applications using deep learning, inference accele...

Xenos: Dataflow-Centric Optimization to Accelerate Model Inference on Edge Devices

Edge computing has been emerging as a popular scenario for model inferen...

A Blockchain Framework for Secure Task Sharing in Multi-access Edge Computing

In the context of Multi-access Edge Computing (MEC), the task sharing me...

Attention-based Feature Compression for CNN Inference Offloading in Edge Computing

This paper studies the computational offloading of CNN inference in devi...

Security Analysis of Capsule Network Inference using Horizontal Collaboration

The traditional convolution neural networks (CNN) have several drawbacks...