NEUKONFIG: Reducing Edge Service Downtime When Repartitioning DNNs

by   Ayesha Abdul Majeed, et al.

Deep Neural Networks (DNNs) may be partitioned across the edge and the cloud to improve the performance efficiency of inference. DNN partitions are determined based on operational conditions such as network speed. When operational conditions change DNNs will need to be repartitioned to maintain the overall performance. However, repartitioning using existing approaches, such as Pause and Resume, will incur a service downtime on the edge. This paper presents the NEUKONFIG framework that identifies the service downtime incurred when repartitioning DNNs and proposes approaches for reducing edge service downtime. The proposed approaches are based on 'Dynamic Switching' in which, when the network speed changes and given an existing edge-cloud pipeline, a new edge-cloud pipeline is initialised with new DNN partitions. Incoming inference requests are switched to the new pipeline for processing data. Two dynamic switching scenarios are considered: when a second edge-cloud pipeline is always running and when a second pipeline is only initialised when the network speed changes. Experimental studies are carried out on a lab-based testbed to demonstrate that Dynamic Switching reduces the downtime by at least an order of magnitude when compared to a baseline using Pause and Resume that has a downtime of 6 seconds. A trade-off in the edge service downtime and memory required is noted. The Dynamic Switching approach that requires the same amount of memory as the baseline reduces the edge service downtime to 0.6 seconds and to less than 1 millisecond in the best case when twice the amount of memory as the baseline is available.


page 2

page 7

page 9


A Case For Adaptive Deep Neural Networks in Edge Computing

Edge computing offers an additional layer of compute infrastructure clos...

Dynamic DNN Decomposition for Lossless Synergistic Inference

Deep neural networks (DNNs) sustain high performance in today's data pro...

Deep neural network based adaptive learning for switched systems

In this paper, we present a deep neural network based adaptive learning ...

Scission: Context-aware and Performance-driven Edge-based Distributed Deep Neural Networks

Partitioning and distributing deep neural networks (DNNs) across end-dev...

DNNShifter: An Efficient DNN Pruning System for Edge Computing

Deep neural networks (DNNs) underpin many machine learning applications....

Accelerating Training of Deep Neural Networks via Sparse Edge Processing

We propose a reconfigurable hardware architecture for deep neural networ...

Parallel and I/O-efficient Randomisation of Massive Networks using Global Curveball Trades

Graph randomisation is an important task in the analysis and synthesis o...

Please sign up or login with your details

Forgot password? Click here to reset