DeepAI AI Chat
Log In Sign Up

Compacting Deep Neural Networks for Internet of Things: Methods and Applications

03/20/2021
by   Ke Zhang, et al.
0

Deep Neural Networks (DNNs) have shown great success in completing complex tasks. However, DNNs inevitably bring high computational cost and storage consumption due to the complexity of hierarchical structures, thereby hindering their wide deployment in Internet-of-Things (IoT) devices, which have limited computational capability and storage capacity. Therefore, it is a necessity to investigate the technologies to compact DNNs. Despite tremendous advances in compacting DNNs, few surveys summarize compacting-DNNs technologies, especially for IoT applications. Hence, this paper presents a comprehensive study on compacting-DNNs technologies. We categorize compacting-DNNs technologies into three major types: 1) network model compression, 2) Knowledge Distillation (KD), 3) modification of network structures. We also elaborate on the diversity of these approaches and make side-by-side comparisons. Moreover, we discuss the applications of compacted DNNs in various IoT applications and outline future directions.

READ FULL TEXT

page 3

page 17

page 25

04/18/2020

Efficient Synthesis of Compact Deep Neural Networks

Deep neural networks (DNNs) have been deployed in myriad machine learnin...
07/11/2022

Sparsifying Binary Networks

Binary neural networks (BNNs) have demonstrated their ability to solve c...
08/20/2019

Efficient Deep Neural Networks

The success of deep neural networks (DNNs) is attributable to three fact...
12/21/2021

Compact Multi-level Sparse Neural Networks with Input Independent Dynamic Rerouting

Deep neural networks (DNNs) have shown to provide superb performance in ...
05/20/2016

Functional Hashing for Compressing Neural Networks

As the complexity of deep neural networks (DNNs) trend to grow to absorb...
10/06/2020

A Panda? No, It's a Sloth: Slowdown Attacks on Adaptive Multi-Exit Neural Network Inference

Recent increases in the computational demands of deep neural networks (D...
02/25/2021

Statistical Testing for Efficient Out of Distribution Detection in Deep Neural Networks

Commonly, Deep Neural Networks (DNNs) generalize well on samples drawn f...