ResPerfNet: Deep Residual Learning for Regressional Performance Modeling of Deep Neural Networks

12/03/2020
by   Chuan-Chi Wang, et al.
0

The rapid advancements of computing technology facilitate the development of diverse deep learning applications. Unfortunately, the efficiency of parallel computing infrastructures varies widely with neural network models, which hinders the exploration of the design space to find high-performance neural network architectures on specific computing platforms for a given application. To address such a challenge, we propose a deep learning-based method, ResPerfNet, which trains a residual neural network with representative datasets obtained on the target platform to predict the performance for a deep neural network. Our experimental results show that ResPerfNet can accurately predict the execution time of individual neural network layers and full network models on a variety of platforms. In particular, ResPerfNet achieves 8.4 absolute percentage error for LeNet, AlexNet and VGG16 on the NVIDIA GTX 1080Ti, which is substantially lower than the previously published works.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/13/2019

Neural network models and deep learning - a primer for biologists

Originally inspired by neurobiology, deep neural network models have bec...
research
06/18/2021

Residual Error: a New Performance Measure for Adversarial Robustness

Despite the significant advances in deep learning over the past decade, ...
research
06/18/2019

Barron Spaces and the Compositional Function Spaces for Neural Network Models

One of the key issues in the analysis of machine learning models is to i...
research
12/01/2020

Toward Accurate Platform-Aware Performance Modeling for Deep Neural Networks

In this paper, we provide a fine-grain machine learning-based method, Pe...
research
02/11/2015

Large-Scale Deep Learning on the YFCC100M Dataset

We present a work-in-progress snapshot of learning with a 15 billion par...
research
01/15/2021

DeepWaste: Applying Deep Learning to Waste Classification for a Sustainable Planet

Accurate waste disposal, at the point of disposal, is crucial to fightin...
research
12/05/2022

Deep Learning Architectures for FSCV, a Comparison

We examined multiple deep neural network (DNN) architectures for suitabi...

Please sign up or login with your details

Forgot password? Click here to reset