Towards Real-Time DNN Inference on Mobile Platforms with Model Pruning and Compiler Optimization

04/22/2020
by   Wei Niu, et al.
0

High-end mobile platforms rapidly serve as primary computing devices for a wide range of Deep Neural Network (DNN) applications. However, the constrained computation and storage resources on these devices still pose significant challenges for real-time DNN inference executions. To address this problem, we propose a set of hardware-friendly structured model pruning and compiler optimization techniques to accelerate DNN executions on mobile devices. This demo shows that these optimizations can enable real-time mobile execution of multiple DNN applications, including style transfer, DNN coloring and super resolution.

READ FULL TEXT
research
01/01/2020

PatDNN: Achieving Real-Time DNN Execution on Mobile Devices with Pattern-based Weight Pruning

With the emergence of a spectrum of high-end mobile devices, many applic...
research
07/04/2022

CPrune: Compiler-Informed Model Pruning for Efficient Target-Aware DNN Execution

Mobile devices run deep learning models for various purposes, such as im...
research
05/02/2019

26ms Inference Time for ResNet-50: Towards Real-Time Execution of all DNNs on Smartphone

With the rapid emergence of a spectrum of high-end mobile devices, many ...
research
01/20/2020

An Image Enhancing Pattern-based Sparsity for Real-time Inference on Mobile Devices

Weight pruning has been widely acknowledged as a straightforward and eff...
research
09/04/2020

CLEANN: Accelerated Trojan Shield for Embedded Neural Networks

We propose CLEANN, the first end-to-end framework that enables online mi...
research
06/18/2022

Augmented Imagefication: A Data-driven Fault Detection Method for Aircraft Air Data Sensors

In this paper, a novel data-driven approach named Augmented Imageficatio...
research
06/20/2023

Exploring the Performance and Efficiency of Transformer Models for NLP on Mobile Devices

Deep learning (DL) is characterised by its dynamic nature, with new deep...

Please sign up or login with your details

Forgot password? Click here to reset