DeepAI
Log In Sign Up

Efficient Deep Neural Networks

08/20/2019
by   Bichen Wu, et al.
25

The success of deep neural networks (DNNs) is attributable to three factors: increased compute capacity, more complex models, and more data. These factors, however, are not always present, especially for edge applications such as autonomous driving, augmented reality, and internet-of-things. Training DNNs requires a large amount of data, which is difficult to obtain. Edge devices such as mobile phones have limited compute capacity, and therefore, require specialized and efficient DNNs. However, due to the enormous design space and prohibitive training costs, designing efficient DNNs for different target devices is challenging. So the question is, with limited data, compute capacity, and model complexity, can we still successfully apply deep neural networks? This dissertation focuses on the above problems and improving the efficiency of deep neural networks at four levels. Model efficiency: we designed neural networks for various computer vision tasks and achieved more than 10x faster speed and lower energy. Data efficiency: we developed an advanced tool that enables 6.2x faster annotation of a LiDAR point cloud. We also leveraged domain adaptation to utilize simulated data, bypassing the need for real data. Hardware efficiency: we co-designed neural networks and hardware accelerators and achieved 11.6x faster inference. Design efficiency: the process of finding the optimal neural networks is time-consuming. Our automated neural architecture search algorithms discovered, using 421x lower computational cost than previous search methods, models with state-of-the-art accuracy and efficiency.

READ FULL TEXT

page 15

page 16

page 18

page 21

page 40

04/25/2022

Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications

Deep neural networks (DNNs) have achieved unprecedented success in the f...
03/20/2021

Compacting Deep Neural Networks for Internet of Things: Methods and Applications

Deep Neural Networks (DNNs) have shown great success in completing compl...
04/04/2017

DyVEDeep: Dynamic Variable Effort Deep Neural Networks

Deep Neural Networks (DNNs) have advanced the state-of-the-art in a vari...
06/30/2020

Boosting Deep Neural Networks with Geometrical Prior Knowledge: A Survey

While Deep Neural Networks (DNNs) achieve state-of-the-art results in ma...
11/03/2017

Accelerating Training of Deep Neural Networks via Sparse Edge Processing

We propose a reconfigurable hardware architecture for deep neural networ...
06/11/2019

Automatic Model Parallelism for Deep Neural Networks with Compiler and Hardware Support

The deep neural networks (DNNs) have been enormously successful in tasks...
01/23/2020

Chameleon: Adaptive Code Optimization for Expedited Deep Neural Network Compilation

Achieving faster execution with shorter compilation time can foster furt...

Code Repositories

pwc

Papers with code. Sorted by stars. Updated weekly.


view repo

IGCV3

Code and Pretrained model for IGCV3


view repo

IGCV3

Code and Pretrained model for IGCV3


view repo

EdgeSpeechNets

implementation of EdgeSpeechNets: Highly Efficient Deep Neural Networks for Speech Recognition on the Edge


view repo