Distributed Deep Neural Networks over the Cloud, the Edge and End Devices

by   Surat Teerapittayanon, et al.

We propose distributed deep neural networks (DDNNs) over distributed computing hierarchies, consisting of the cloud, the edge (fog) and end devices. While being able to accommodate inference of a deep neural network (DNN) in the cloud, a DDNN also allows fast and localized inference using shallow portions of the neural network at the edge and end devices. When supported by a scalable distributed computing hierarchy, a DDNN can scale up in neural network size and scale out in geographical span. Due to its distributed nature, DDNNs enhance sensor fusion, system fault tolerance and data privacy for DNN applications. In implementing a DDNN, we map sections of a DNN onto a distributed computing hierarchy. By jointly training these sections, we minimize communication and resource usage for devices and maximize usefulness of extracted features which are utilized in the cloud. The resulting system has built-in support for automatic sensor fusion and fault tolerance. As a proof of concept, we show a DDNN can exploit geographical diversity of sensors to improve object recognition accuracy and reduce communication cost. In our experiment, compared with the traditional method of offloading raw sensor data to be processed in the cloud, DDNN locally processes most sensor data on end devices while achieving high accuracy and is able to reduce the communication cost by a factor of over 20x.


page 7

page 8


Cost-Driven Offloading for DNN-based Applications over Cloud, Edge and End Devices

Currently, deep neural networks (DNNs) have achieved a great success in ...

Hierarchical Training of Deep Neural Networks Using Early Exiting

Deep neural networks provide state-of-the-art accuracy for vision tasks ...

Privacy Aware Offloading of Deep Neural Networks

Deep neural networks require large amounts of resources which makes them...

Guardians of the Deep Fog: Failure-Resilient DNN Inference from Edge to Cloud

Partitioning and distributing deep neural networks (DNNs) over physical ...

FTPipeHD: A Fault-Tolerant Pipeline-Parallel Distributed Training Framework for Heterogeneous Edge Devices

With the increased penetration and proliferation of Internet of Things (...

Generative Sensing: Transforming Unreliable Sensor Data for Reliable Recognition

This paper introduces a deep learning enabled generative sensing framewo...

Partitioning and Deployment of Deep Neural Networks on Edge Clusters

Edge inference has become more widespread, as its diverse applications r...

Please sign up or login with your details

Forgot password? Click here to reset