OODIn: An Optimised On-Device Inference Framework for Heterogeneous Mobile Devices

06/08/2021
by   Stylianos I. Venieris, et al.
0

Radical progress in the field of deep learning (DL) has led to unprecedented accuracy in diverse inference tasks. As such, deploying DL models across mobile platforms is vital to enable the development and broad availability of the next-generation intelligent apps. Nevertheless, the wide and optimised deployment of DL models is currently hindered by the vast system heterogeneity of mobile devices, the varying computational cost of different DL models and the variability of performance needs across DL applications. This paper proposes OODIn, a framework for the optimised deployment of DL apps across heterogeneous mobile devices. OODIn comprises a novel DL-specific software architecture together with an analytical framework for modelling DL applications that: (1) counteract the variability in device resources and DL models by means of a highly parametrised multi-layer design; and (2) perform a principled optimisation of both model- and system-level parameters through a multi-objective formulation, designed for DL inference apps, in order to adapt the deployment to the user-specified performance requirements and device capabilities. Quantitative evaluation shows that the proposed framework consistently outperforms status-quo designs across heterogeneous devices and delivers up to 4.3x and 3.5x performance gain over highly optimised platform- and model-aware designs respectively, while effectively adapting execution to dynamic changes in resource availability.

READ FULL TEXT

page 1

page 2

page 4

research
01/13/2021

An Empirical Study on Deployment Faults of Deep Learning Based Mobile Applications

Deep Learning (DL) is finding its way into a growing number of mobile so...
research
11/08/2018

When Mobile Apps Going Deep: An Empirical Study of Mobile Deep Learning

Deep learning (DL) is a game-changing technique in mobile scenarios, as ...
research
06/01/2023

ModelObfuscator: Obfuscating Model Information to Protect Deployed ML-based Systems

More and more edge devices and mobile apps are leveraging deep learning ...
research
03/11/2021

Integration of Convolutional Neural Networks in Mobile Applications

When building Deep Learning (DL) models, data scientists and software en...
research
04/20/2021

DynO: Dynamic Onloading of Deep Neural Networks from Cloud to Device

Recently, there has been an explosive growth of mobile and embedded appl...
research
06/05/2019

Adroitness: An Android-based Middleware for Fast Development of High-performance Apps

As smartphones become increasingly more powerful, a new generation of hi...
research
12/27/2022

A systematic literature review on the development and use of mobile learning (web) apps by early adopters

Surveys in mobile learning developed so far have analysed in a global wa...

Please sign up or login with your details

Forgot password? Click here to reset