Real-Time Video Inference on Edge Devices via Adaptive Model Streaming

06/11/2020
by   Mehrdad Khani, et al.
5

Real-time video inference on compute-limited edge devices like mobile phones and drones is challenging due to the high computation cost of Deep Neural Network models. In this paper we propose Adaptive Model Streaming (AMS), a cloud-assisted approach to real-time video inference on edge devices. The key idea in AMS is to use online learning to continually adapt a lightweight model running on an edge device to boost its performance on the video scenes in real-time. The model is trained in a cloud server and is periodically sent to the edge device. We discuss the challenges of online learning for video and present a practical design that takes into account the edge device, cloud server, and network bandwidth resource limitations. On the task of video semantic segmentation, our experimental results show 5.1–17.0 percent mean Intersection-over-Union improvement compared to a pre-trained model on several real-world videos. Our prototype can perform video segmentation at 30 frames-per-second with 40 milliseconds camera-to-label latency on a Samsung Galaxy S10+ mobile phone, using less than 400Kbps uplink and downlink bandwidth on the device.

READ FULL TEXT

page 3

page 14

research
10/04/2019

Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing

As a key technology of enabling Artificial Intelligence (AI) application...
research
04/26/2022

AccMPEG: Optimizing Video Encoding for Video Analytics

With more videos being recorded by edge sensors (cameras) and analyzed b...
research
02/02/2021

Autodidactic Neurosurgeon: Collaborative Deep Inference for Mobile Edge Intelligence via Online Learning

Recent breakthroughs in deep learning (DL) have led to the emergence of ...
research
08/31/2023

Edge-Assisted On-Device Model Update for Video Analytics in Adverse Environments

While large deep neural networks excel at general video analytics tasks,...
research
03/28/2022

DNN-Driven Compressive Offloading for Edge-Assisted Semantic Video Segmentation

Deep learning has shown impressive performance in semantic segmentation,...
research
08/24/2022

Efficient Heterogeneous Video Segmentation at the Edge

We introduce an efficient video segmentation system for resource-limited...
research
03/21/2020

Edge-assisted Viewport Adaptive Scheme for real-time Omnidirectional Video transmission

Omnidirectional applications are immersive and highly interactive, which...

Please sign up or login with your details

Forgot password? Click here to reset