Creating Robust Deep Neural Networks With Coded Distributed Computing for IoT Systems

04/09/2021
by   Ramyad Hadidi, et al.
0

The increasing interest in serverless computation and ubiquitous wireless networks has led to numerous connected devices in our surroundings. Among such devices, IoT devices have access to an abundance of raw data, but their inadequate resources in computing limit their capabilities. Specifically, with the emergence of deep neural networks (DNNs), not only is the demand for the computing power of IoT devices increasing but also privacy concerns are pushing computations towards the edge. To overcome inadequate resources, several studies have proposed the distribution of work among IoT devices. These promising methods harvest the aggregated computing power of the idle IoT devices in an environment. However, since such a distributed system strongly relies on each device, unstable latencies, and intermittent failures, the common characteristics of IoT devices and wireless networks, cause high recovery overheads. To reduce this overhead, we propose a novel robustness method with a close-to-zero recovery latency for DNN computations. Our solution never loses a request or spends time recovering from a failure. To do so, first, we analyze the underlying matrix-matrix computations affected by distribution. Then, we introduce a new coded distributed computing (CDC) method that has a constant cost with the increasing number of devices, unlike the linear cost of modular redundancies. Moreover, our method is applied in the library level, without requiring extensive changes to the program, while still ensuring a balanced work assignment during distribution. To illustrate our method, we perform experiments with distributed systems comprising up to 12 Raspberry Pis.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/08/2019

Collaborative Execution of Deep Neural Networks on Internet of Things Devices

With recent advancements in deep neural networks (DNNs), we are able to ...
research
10/25/2020

DistPrivacy: Privacy-Aware Distributed Deep Neural Networks in IoT surveillance systems

With the emergence of smart cities, Internet of Things (IoT) devices as ...
research
09/29/2019

Futures-Based Resource Trading and Fair Pricing in Real-Time IoT Networks

Technology advancements on sensing, communications, and computing direct...
research
06/21/2023

Design of Energy Harvesting based Hardware for IoT Applications

Internet of Things (IoT) devices are rapidly expanding in many areas, in...
research
10/06/2021

FTPipeHD: A Fault-Tolerant Pipeline-Parallel Distributed Training Framework for Heterogeneous Edge Devices

With the increased penetration and proliferation of Internet of Things (...
research
12/11/2021

Automated Customization of On-Thing Inference for Quality-of-Experience Enhancement

The rapid uptake of intelligent applications is pushing deep learning (D...
research
04/04/2018

Personal Volunteer Computing

Since the 1990s, the number of personal computing devices has exploded. ...

Please sign up or login with your details

Forgot password? Click here to reset