Scheduling Real-time Deep Learning Services as Imprecise Computations

11/02/2020
by   Shuochao Yao, et al.
5

The paper presents an efficient real-time scheduling algorithm for intelligent real-time edge services, defined as those that perform machine intelligence tasks, such as voice recognition, LIDAR processing, or machine vision, on behalf of local embedded devices that are themselves unable to support extensive computations. The work contributes to a recent direction in real-time computing that develops scheduling algorithms for machine intelligence tasks with anytime prediction. We show that deep neural network workflows can be cast as imprecise computations, each with a mandatory part and (several) optional parts whose execution utility depends on input data. The goal of the real-time scheduler is to maximize the average accuracy of deep neural network outputs while meeting task deadlines, thanks to opportunistic shedding of the least necessary optional parts. The work is motivated by the proliferation of increasingly ubiquitous but resource-constrained embedded devices (for applications ranging from autonomous cars to the Internet of Things) and the desire to develop services that endow them with intelligence. Experiments on recent GPU hardware and a state of the art deep neural network for machine vision illustrate that our scheme can increase the overall accuracy by 10

READ FULL TEXT

page 2

page 3

page 4

page 6

page 7

page 8

page 9

page 10

research
05/05/2019

Zygarde: Time-Sensitive On-Device Deep Intelligence on Intermittently-Powered Systems

In this paper, we propose a time-, energy-, and accuracy-aware schedulin...
research
10/02/2018

Cloud Chaser: Real Time Deep Learning Computer Vision on Low Computing Power Devices

Internet of Things(IoT) devices, mobile phones, and robotic systems are ...
research
02/02/2021

TinyML for Ubiquitous Edge AI

TinyML is a fast-growing multidisciplinary field at the intersection of ...
research
05/05/2023

Tiny-PPG: A Lightweight Deep Neural Network for Real-Time Detection of Motion Artifacts in Photoplethysmogram Signals on Edge Devices

Photoplethysmogram (PPG) signals are easily contaminated by motion artif...
research
05/09/2019

Deep Learning Acceleration Techniques for Real Time Mobile Vision Applications

Deep Learning (DL) has become a crucial technology for Artificial Intell...
research
02/10/2020

Pairwise Neural Networks (PairNets) with Low Memory for Fast On-Device Applications

A traditional artificial neural network (ANN) is normally trained slowly...
research
11/05/2022

A review of TinyML

In this current technological world, the application of machine learning...

Please sign up or login with your details

Forgot password? Click here to reset